Remember back on Monday and Tuesday when this blog and several others were cautioning everyone to ignore leaked early exit polls?
I figured I should go back and check to see whether that advise turned out to be sound or not. Quick answer: yes. Ignoring the exit polls and using the pre-election polls gave a much better indication of what was happening.
One thing that's changed is that we no longer get early exits leaked as early as we used to. In the old days, insiders would hear about exit polls in midday, reflecting the earliest wave of polling. Now, the networks really don't allow anything to leak until much later; of course, when it does leak, everyone with a computer hears about it.
So when they did leak in late afternoon or early evening (and Taegan Goddard's wonderful Political Wire reported them), how useful were they?
The leak, as Goddard reported, included 11 states.
First cut: compared to nothing at all, the leaked exit polls (LEPs) did okay. They reported leads in 9 states, and only had one wrong: the LEPs had Romney up one in Florida, but Obama won it by 0.6. The LEPs were within three points in all but two states. That seems at least somewhat useful. However, there was also a solid pro-Romney bias in the LEPs; In nine of the eleven states, the LEPs had numbers better for Romney than the actual results.
Second cut: Add all that up, and the LEPs gave the mistaken impression of a narrow and not certain Obama win. With Florida leaning Romney and Virginia and Colorado tied, the LEPs had the election Obama 281, Romney 235, undecided 22. With Ohio one of the two states where the LEPs favored Obama (showing a four point lead, while it wound up only two), there was no obvious way for Romney to get the rest of the way, but a whole bunch of targets: he was losing in the LEPs by three in Minnesota, Iowa, and New Hampshire, and by four in Ohio, Pennsylvania, Wisconsin, and Minnesota. For what it's worth, there were no LEPs, at least in this set, for Michigan, New Mexico, or Oregon. So what you're left with if you were hoping for Romney was that the supposedly tied states would go his way, and that somehow he would find another 13 EVs. Unlikely, but perhaps slightly more likely than what the polling aggregators and predictors had been saying.
Third cut: but that's when we get to why these LEPs were actually totally useless. Going back to HuffPollster's final polling averages, the LEPs beat pre-election polls in only New Hampshire and Nevada, and New Hampshire only by a bit (Obama's lead was 3 in LEP, 2.5 in HuffPollster; the vote was a 6 point win). In three other states, LEP and HuffPollster were tied. So in six of eleven states, using the pre-election polls was better than using the leaked exit polls. Or, if we want to look at the simple average miss, the LEPs averaged miss was 2.8, while HuffPollster in these states averaged 2.2.
So, yeah, remember it for next time: ignore leaked early exit polls.
Friday, November 9, 2012
Subscribe to:
Post Comments (Atom)
Posts
So exit polls were heavily biased toward Kerry in 2004; slightly biased toward Romney in 2012.
ReplyDeleteIs there something systematic about these polls causing them to underestimate the standing of the incumbent seeking re-election?
Or is it just a matter of sampling error? After all, an exit poll is just a poll; it has the same limitations as any other single poll.
I believe the exit polls are also subject to their own form of 'house effect.' Obviously likely-voter screens are not an issue, but exit polling is done at a necessarily limited number of individual precincts. These are chosen to be representative of larger areas (states? counties?), modeled using previous elections and assumed demographic trends. If the models are off ... oops.
DeleteThe big picture seems to confirm this. The exit polls notoriously had a Dem bias in 2000 and 2004 - building lots of false hopes. :-( I think they may have had in 2008 as well, without the drastic hangover effect. And it sounds like the exit poll model builders over-corrected a bit this time, choosing precincts/adjustments that overstated Romney's vote.
Which won't keep me from looking for exit numbers next time, if only to help in reading between the lines of what they're saying on CNN as we wait for the real numbers to come in ...
Well, to the extent that the LEPs gave Republicans some faint encouragement -- magnifying the pain and shock of their later defeat -- I say hurrah for exit polls.
ReplyDeleteIt was only fair, after 2004.
From your lips to Karl Rove's ears! (Or in the modern age, do I need to say fingers to eyes?)
ReplyDeleteSecond cut: Add all that up, and the LEPs gave the mistaken impression of a narrow and not certain Obama win
ReplyDeleteThis renders odd Romney's reticence to take the stage at midnight, no? If we assume the LEPs were exactly correct (since, ceterus paribus, the most likely outcome for a normal distribution is always the observed one), then by noon Team Romney, and Mitt himself, must have known their probable result was a close loss with a few reasonable, only-slightly-white-knuckle roads to victory.
I get the whole argument that supercommitment causes one to think there's no way he could lose. But c'mon man! We were tired in the eastern time zone, and we still had to hear from Obama! Losing a Presidential election is surely difficult to come to terms with, but 12 hours must be enough to drag yourself out to the podium. In fact, 11 will probably do.
Seems to me that early exit polls would reflect seniors voting earlier in the day, and so would trend Republican, just as early election results give the southern states first and trend Republican.
ReplyDelete