Elsewhere

I know there hasn't been much here lately, but I've been busy as usual over at PP, plus a bit at Greg's place.

I'm really skeptical about the "Obamacare rollout hurt Obama's approval ratings" story. It just doesn't match the numbers, as far as I can tell. Part of which is that it's a bit difficult to know what to make of the shutdown. Since I'm inclined to think the shutdown hurt Obama's approval, I see less movement from the ACA fiasco than those who believe the shutdown helped him.

Here's some others from the last week or so:

Return of the Huck.

A big lesson from the HealthCare.gov fiasco

Polling? Interviews? Just get it right.

No, don’t shut up the minority on judges

More Senate drama

Ignore Those Polls!

The "Millenials" poll out earlier this week showing (among other things) younger people unenthusiastic about the ACA has sparked a fair amount of discussion, but there's one particular point I haven't seen made. I'm highly suspicious of the reported finding that only a third of the uninsured from ages 18 through 29 intend to enroll through the exchanges.

There are just too many things out there for me to believe that there's any kind of predictable relationship between what they tell a pollster and what they'll actually do.

Let's see...first of all, I'm pretty skeptical about asking the younger part of this cohort about personal intentions with regard to health insurance at all. Perhaps I find it hard to put myself in their shoes, since I was a full-time college student at that age and from a middle class family; I didn't have to think about health insurance at all until after graduation. At any rate, for those at ages 18 through 25, the survey asked what they would do "after you are no longer eligible to stay on your parents plan." As far as I can tell from the survey, that was used for everyone in that age range, regardless of whether they had insurance on a parent plan now.  If that's case, then I'd suggest it's totally meaningless what a 18 year old tells a pollster she expects to do about obtaining health insurance over five years into the future.

And then there's the question of what they're signing up for. The poll used a split sample to ask about both "Affordable Care Act" and "Obamacare," which is great. But there's no mention of Healthcare.gov in either case; instead, it refers to only "government-run exchanges." I'm really not confident that most young people know what that means -- or that they would need to in order to sign up. Then there's the issue of the state-run exchanges; people in Kentucky or California may believe that they're not signing up for "Obamacare" at all. It's also true, although again there's no way to know how it affects polling responses, that a fair number of uninsured young people will be eligible for expanded Medicaid. Would they say they are planning on using "government-run exchanges" or not? That's not even figuring out what to think about those who would be eligible for Medicaid but have the bad luck to be in Texas or some other state that didn't participate.

One more thing: even for those who are eligible for the exchanges now and understand what they are, it's still not clear how well what they tell pollsters of their intentions will match what they eventually do. Some may say they intend to sign up but then never get around to it, or start shopping and then decide it's a bad deal for them. Others may not intend to and then find out that subsidies make it a good deal.

My guess is that for most young people, at some point they'll confront the question of "how do I get health insurance?" for the first time -- in many cases, because a parent is nagging them about it -- and unless they have employment-linked coverage, the answer is going to be to go to Healthcare.gov or their state exchange. And, yeah, it won't really feel like signing up for Obamacare; it's just going to be how you get (private) health insurance.

Which, I should add, is not a prediction about how many will sign up and at what age, and how that will affect the overall pool (and thus the degree to which the ACA exchanges work or don't work).

At any rate, I don't think the poll tells us anything about what young people are going to do when they get to that point of seeking insurance. General point: be very suspicious any time people answer polling question about intentions they might not have had until they were asked. Ignore those polls!

One More Time on Subsamples (Ignore Those Polls! Addendum)

Last week I took someone to task for placing too much weight on Gallup's reports of Latino voters supposedly disproportionate turn away from Barack Obama. One week more, and...it's disappearing even more. In the Gallup weekly reports, Obama trickled down to 40% approval overall after three weeks at 41%, but his approval among Hispanic citizens is up for the third week in a row and now stands at 54%, 14 points higher than his overall rating. That's consistent with where he was back in September (that is, it's down, but not down more than the overall dip).

It's certainly possible that there's something going on here, but it's also possible, and perhaps a bit more likely, that the earlier dip never actually happened. It was just a bit of random variation.

I wanted to post on this not to beat up on the same thing, but because there's one additional point I didn't mention which is actually pretty important. Gallup publishes, in their weekly reports, a colossal 41 different splits. Look: flip a single coin 1000 times, and if heads comes up 600 times, something is probably going on. But what you're doing here is walking into a coin flipping factory, with some flipped 1000 times, and some 100 times, and some only 10 times, and yeah, if all you are doing are looking for oddball results then you're going to find what you're looking for. 

And so, like clockwork, we have another reporter tweeting out that "Obama's approval has dropped the sharpest among Eastern wealthy post-grads." Give it a rest, folks: just don't trust these blips in the crosstabs. There's no way of knowing whether they're real or not. 

Ignore Those Polls! (Small Sample Size Crosstabs Edition)

Josh Kraushaar also said:

The president isn't just losing his skeptics from the chaotic Obamacare rollout but his allies who stood to gain from the law's benefits -- namely Hispanics, whose approval of the president has dropped more than any demographic subgroup since the problems began. 

Ignore those polls! Or at least, be very careful about subsamples. Kraushaar published this one early today, before last week's Gallup numbers were ready; turns out that Obama's approval among Hispanics, per Gallup, is now at 52% -- up three points over two weeks ago, while the overall rate remained stuck at 41%.

Or whatever. These kinds of subsamples can be all over the place. As it happens, I had noticed the falloff too...and I had been planning to write this item if it turned around, which it more or less has, even before Kraushaar gave me the additional hook.

(Oh, and yes -- the press frenzy of the last two weeks is accompanied by a zero-point decline in approval; Obama is down 2 points overall since the last full week of September, which is the last week before the rollout of the exchanges and the shutdown. Two percentage points. Free fall!).

Here's the difference between Obama's overall approval rating and his rating among Hispanics, going back over time, with this week at 52% overall - 41% Hispanic = 11 points:

11, 9, 8, 15, 13, 10, 14, 10, 16, 14, 16, 15, 18, 18, 17, 20, 18, 17, 18, 18, 19, 19, 20, 15, 20

Going back further, it's around 20 to the beginning of the year, which is as much as Gallup has posted now. So...it does appear that there has probably been a change over time. But the decline started a while ago; that last 20 point gap is at the end of July, and the gap was down to 10 points at the end of September -- before the exchanges, and before the shutdown.

If I had to guess, it would be that it's more about immigration disappearing from the news agenda; as budget and health care dominated the news, Latino voters reacted more or less the same as everyone else. But that's a guess! It's impossible to really get any closer to the truth from these numbers. Other than it's unlikely -- not impossible, but unlikely -- that it's reflecting something about the ACA rollout. The sequence just doesn't seem to suggest that.

The general point: be very, very, very, very wary about using the subsamples in these kinds of polls. It's incredibly easy to get big jumps that are really just random noise -- look, for example, at the 20-15-20 sequence right at the end there. The odds of getting all worked up over something that isn't real are, unfortunately, very high.

Electoral Effects of the...Oh, C'mon, People, Really?

Sometimes, I think it helps to be a solid 1500 miles away from Washington. Or maybe it helps to have read a little history. Or maybe I'm just an old guy.

Anyway: we're at a point at which even Jonathan Chait, writing against panic, says that "the current sense of dread enveloping the Democratic Party has a very real basis. President Obama’s poll numbers are plunging to unprecedented depths."

Yes, it's all massively overstated.

Plunging? His approval (and some associated numbers) have definitely dropped. I suppose "plunge" is subjective, but HuffPollster's estimate, set for "less smoothing" and therefore (over?) sensitive to recent polls, is that he's lost maybe 2.5 percentage points over the last five or so weeks. He's been losing ground all year including, mostly likely, during the shutdown. Depending on what adjustments one does, that might have accelerated after the shutdown, or maybe not. It doesn't sound like a "plunge" to me. 

Obama's popularity is probably at the low point of his presidency (again, depending on the adjustments, he's either a bit below or a bit above his previous low. But it's not any kind of unusually low low point (he's nowhere near Truman, Carter, Nixon, W.), there's no particular reason to expect the slump to continue, and myths aside no reason to believe he won't recover if the news turns better. Granted, it's hard to know what to expect from healthcare.gov, but it's not as if it's getting worse over time. I'm not saying his numbers will go up. Just that it's more or less equally likely as further drops. 

(Actually...if I had to guess, I'd say a run of either stability or improvement is probably more likely, at least if the next budget deadlines come and go quietly. Gallup's economic confidence index has been steadily recovering from its shutdown/debt limit plunge -- yes, that one was a real plunge -- and Jamelle Bouie is right that the economy is a very big part of presidential approval, although I think he somewhat understates the ability of other events to matter).

As for electoral effects? I wrote an item dismissing direct electoral effects of the shutdown against Republicans back last month; that post pretty much works now, in reverse for effects against Democrats. I should say: it's far easier for sentiment against the president to translate into midterm electoral losses than it is for feelings against the out-party. So if Obama is unpopular in November 2014, it will hurt Democrats. But today's frenzy about the ACA is going to be mostly forgotten by then, one way or another, just as the shutdown seems forgotten today. That's probably even true, believe it or not, if the program totally collapses, although I don't think that's going to happen.

Anyway, Obama's approval ratings have in fact fallen from the mid-40s to the low-40s, and over the course of the year from around 50 to the low 40s. It's obviously not good news for him, but it seems a lot less dramatic than a lot of the chatter this week would have it be. 

Does VA Gov Explain the 1992 Perot Vote?

One more time into the question about the VA-Gov polls and the election results.

I came up with a possible (and speculative) explanation based on the condition of two candidates who were not especially popular among their own parties, along with a sufficiently well-publicized third candidate. Perhaps Democrats trusted the polls, believing that the seven-point lead Terry McAuliffe held was real, and therefore believed it was safe to vote for the Libertarian; meanwhile, Republicans didn't trust the polls, believed it was close, and thus switched at the last minute back to the Republican.

It occurs to me there doesn't have to be a difference in partisan bias to get the results I speculated on. Suppose partisans on both sides read the polls with a 5 point bias in their own direction. Then, in a 7 point race with a third party candidate, the winning side party is going to perceive a 12 point landslide and vote their true preference, while the losing side party will perceive a 2 point barnburner and reluctantly return home. After all, a heavily-polled election with a 7 point spread will surely have some 12 point and some 2 point polls, and in fact this one did.

It further occurs to me that if true, this should be a pattern we see in other elections. The main conditions, I would think, would have to be a polling lead in the 5-10 point range...that's small enough that the losing side can plausibly believe that it's very close, but large enough for the winning side to feel safe -- again, if both sides interpret the polling with about the same bias.

All of which gets to that 1992 Perot vote. The research on that was pretty clear: Perot took equally away from both candidates. However, if this (speculated! hypothesized!) effect was operating, then it would have cost Perot Republican votes while leaving Democrats parked with him at the end. If that was the case, then the "real" Perot vote (that is, those who would have voted for him all else equal) would have been more Republican the observed Perot vote.

Note, of course, that it still wouldn't be the case that Perot cost Republicans the 1992 election; no third party candidate, and all those votes come back home, half to Clinton and half to Bush.

At any rate: I suspect this should be testable. The basic idea should be that an effect kicks in when there's a 5-10 point lead and a plausible third party candidate...say, one getting better than 5% of pre-election polls. Under those circumstances, the final result should be more favorable to the losing candidate than the polls predicted. It's also possible, if detailed polls are out there, to compare the third party supporters before election day and in the final vote.

It should also be an effect, if real, that may stronger when the party-aligned press has a larger share of the overall press, since presumably that would produce stronger cues for interpreting the polls with a bias.

Not sure if anyone wants to bother doing the work on this (I don't), but I think there's a fairly good chance that it's real -- both for VA-Gov and for Perot.

Ignore Those Polls! (Shutdown Edition)

There are polls out that have people opposing a shutdown over the ACA; there's also at least one poll out that has blame split if there is a shutdown.

Ignore those polls!

The later -- "who would you blame?" type polls -- are particularly not to be trusted. People aren't very good at predicting their own reactions to things, especially things they don't know much about and haven't thought much about. But I'd be somewhat skeptical of the first set, as well. It's one thing for (even) Republicans say they are against a description of the Ted Cruz/Mike Lee shutdown strategy; it's quite another for them to oppose it once it happens. Perhaps they will, but it's just not the same thing.

Here's a couple of rule-of-thumb type advice that I use on these sorts of things. Is the question one that respondents, especially those who are not political junkies, actually would have solid opinions about absent a pollster phone call? A high-profile election next week; sure. A budget deal, or a budget negotiations scenario? Far less likely. And then: does the question ask respondents for their current reactions to something, or to predict future reactions?

On questions such as these, which violate both of those conditions, I think it's far better to treat the polling at best as a vague indication of vague first thoughts. At best. Meanwhile, to predict the actual reaction (at least in polling) to future events, it's much wiser to focus on how information flows are likely to develop from those events; that's what I was doing in my TAP column about Life During Shutdown. Granted: such analysis requires political judgement, and can easily be wrong. But in my view, that's still a better bet than trusting what people say now how they think they'll react in the future. 

Catch of the Day

You probably read about that PPP poll result in which Louisiana Republicans blamed Barack Obama for Katrina. Great fun for liberals! But do read Mark Blumenthal and Ariel Edwards-Levy for a round up of coverage and their analysis of the question, especially this:

The problem with this question should be obvious: It was asked only of Republicans and offered just two choices: Bush, who was president at the time and Obama, who was still a junior senator from Illinois. "If you ask the vast majority of Louisiana Republicans to identify the leader most responsible for the Katrina response," the conservative Louisiana blog The Hayride argued, "you’re going to get Kathleen Blanco and Ray Nagin as the answer. Replacing those two on the ballot with Barack Obama so that the only choices available are Bush, Obama or 'not sure' is malpractice, and mean-spirited malpractice at that....The question was designed to force Louisiana Republicans to trash the former president in “It’s Bush’s fault” fashion. And that was transparently obvious to the respondents. Anybody could see what they were trying to do. So 72 percent of those polled basically told PPP to do something anatomically impossible. 44 percent chose the most non-responsive response they could, which was to say 'not sure.' And the other 29 percent went even further and gave the in-your-face answer of Obama.

Here's what you need to remember about surveys: they don't primarily report opinions, or predicted action; they report how people answer the specific questions asked in the circumstances under which they are asked.

In many cases, the way people answer questions from a pollster does in fact have a close relationship with their opinions; in many cases, it can predict action. In plenty of other cases, however, that's just not the case.

For a common sense rule of thumb...the closer a question is to something that respondents have been actually thinking about, the better the chance of the answer indicating something we would normally think of as a real, stable, opinion. The closer a question is to something that someone is already planning to do, the better it will actually predict actions. So, in the classic case, asking someone who they will vote for tomorrow is going to be an excellent way of finding out election results in advance; asking someone who they would vote for if an election scheduled three years from now was to take place today is going to produce a very poor prediction of future actions.

So, yeah, feel free to have fun with this sort of thing...I have no problem with PPP for doing it. Just be careful not to read it too literally or take it very seriously.

And: nice catch!

Tweeting and Elections

I have a TAP column out today in which I'm not very kind to the sociologists who got a ton of publicity last week from a claim that "tweet share" could predict elections. I only had access when I was writing to an early version of the paper, and I wasn't impressed...but the bulk of my item was really about what polling and survey research is good for, based on one of the authors unimpressive op-ed which made implausible claims about the technique.

Blog posts here go up, usually, as soon as I write them (although every once in a while I'll write in advance and hold it, or sometimes during the day I'll hold one for a while just to keep a better flow). Blog posts at WaPo usually take half an hour or so, and it's not unusual for them to sit an hour or two. Columns are different...I almost always write the Saturday Salon column on Friday, while TAP columns sometimes take a while to go through editing, and then another wait sometimes for them to slot it in where they want. All of which is to say: since I wrote the this one, some excellent critiques of the study itself are out.

So if you're interested in the study, please read Jonathan Nagler over at the Monkey Cage, who casts quite a bit of doubt on the findings. That's the best one, but you may also want to read Rob Santos over at The Fix. And Mark Blumenthal and Ariel Edwards-Levy are also very good at HuffPollster. Add it all up, and what you get is this: the study is a dud, and the op-ed publicizing it made claims that the study didn't even support, even if it had been solid.

Fortunately, my piece is just fine as is, even without all of that (well, I suppose it's not for me to say it's fine; I just mean it didn't depend on the tweet study actually being any good). That's because I'm primarily using the tweet study as an excuse, as I said, to discuss what polling is good for. Which is quite a lot! I know I'm always saying that people should ignore this or that poll, but that doesn't mean polling and survey research is worthless. Not at all. So if you want a short primer about what polling is good for anyway, that's my column.

Read Stuff, You Should

Happy Birthday to Michelle Phillips, 69. Going with the imdb link instead of allmusic; I do like the Mamas and the Papas, but I like her Star Trek episode more.

And on to the good stuff:


1. David Frum's blog will be missed as he takes the summer off, or more. His sign-off, an honest report on the state of conservative reformers, is excellent.

2. Jared Bernstein is keeping up the sequestration watch.

3. Jonathan Cohn gets upset with conservatives. With one conservative. It's about ACA, natch.

4. Brendan Nyhan on scandalmania and Obama's approval ratings.

5. The death of the "gay recruiting myth," from E.J. Graff.

6. And Peter Gammons on tired baseball.

What Mattered This Week?

I haven't mentioned it for a while, I don't think, but I do still believe the war in Syria matters, so I'll go with that this time.

For something that doesn't matter...not sure. I did write two Ignore Those Polls! posts this week (ACA and scandals), so I'll go with that, I suppose.

That doesn't seem like a very good job, so I hope y'all have better: what do you have? What do you think mattered this week?

Ignore Those Polls! (Scandals and the Economy Edition)

There's a new Quinnipiac poll out with findings about scandalmania that are drawing attention, including a ranking of the scandals (respondents answered that the IRS scandal is more important than Benghazi or the AP stories) and something that sounds very nice at the moment for Democrats: by a large margin, respondents say that the economy is more important than the scandals.

Ignore those polls!

First: people are notoriously terrible about predicting how events will affect their votes, or even their future opinions. People may say that the economy is more important than scandals...but if scandals are in the news constantly, people will pay attention, and they could well affect vote choice even if in some abstract sense voters don't "think" they are important. What's more, constant news coverage -- if it continues -- will strongly signal that a scandal is important, and people will respond accordingly.

Second: the polling that really tells us something about voters and the scandals so far is already out, with the president's approval ratings holding steady over the last month (actually, Pollster's average has had Obama's approval ratings on a slow decline since the beginning of the year, but setting it to "less smoothing" shows that decline has decelerated since mid-March, and certainly no apparent new movement since the IRS thing broke). Here, the Q poll has a low Obama approval number of 45%, down three ticks from May 1, but as always the polling averages are more useful, and they've already told as that nothing's happening.

Third: So we know that unlike, say, Iran-Contra, the current scandalmania is not having a significant direct effect on the president's popularity. But we also know that absent new revelations, each of these stories will be ancient and forgotten by the time voters engage in the 2014 election cycle. Really, then, what's needed is good guesses about the scandals themselves, and voters aren't in a particularly good position to make those guesses.

In other words, if it turns out that Barack Obama and Hillary Clinton secretly worked with zombie bin Laden to plan and carry out Benghazi, then nothing voters say now about the IRS story being more important, or the economy trumping any of the scandals, will matter. If on the other hand Benghazi continues to be about parsing the difference between terror attacks and terrorist attacks...well, we've long known that no one apart from dedicated partisans cares about it.

All in all, other than tossing another number into the approval averages, I think we're safe ignoring this stuff. Given that scandalmania isn't doing anything significant now, the way that people rank the individual stories or how they compare them to other vote factors just isn't telling us anything.

Catch of the Day

To Mark Blumenthal and Emily Swanson, who recreate the classic "1975 Public Affairs Act" survey and find -- no surprise -- that lots of people are very willing to share their opinions about a fictional law when they get asked, and that they'll even fall into familiar partisan reactions if properly prompted about fictional political controversies.

It's a great item, and lots of fun.

I'll just add one important thing: those of us who are constantly harping on this stuff are not saying that voters are stupid. I don't believe that voters are stupid! Well, I suppose some of them, but...no, really, I don't. It's all just about two things: knowing that most people don't pay a whole lot of attention to politics, and knowing that we shouldn't take polling results at face value -- especially when it concerns things that most people don't may much attention to.

Look: when people's immediate interests are affected and they get engaged in politics, they often are quite sensible about it. But for most of us, most of the time, that's not the case.

Now, I think a lot of people who ignore politics, a lot of people who cynically dismiss the possibility of effective political action, a lot of people who think that only big shots can make any difference...I think they would find not only that individuals can make a difference, but that they would really find a sense of meaning and fulfillment if they got involved (what the American founders called "public happiness"). And I'm with Machievelli, Arendt, and others, including in my view Madison, in believing that one of the real reasons to have a republic is to allow access to that sense to every single citizen.

But I'm also willing to accept the facts, which is that most people don't do that. And that's fine, too. Fortunately, it turns out that democracy works well enough even if most citizens do little more than vote and use shortcuts such as party even to do that. And there's nothing insulting or dismissive in accepting that they do that. Nor is there anything insulting or dismissive in realizing that people give survey answers which shouldn't be taken at face value because they barely pay attention to politics.

At any rate: great catch!

Hey, Pollsters: What's "Waste"?

I recommend to everyone an excellent Mark Blumenthal summary of the polling on government spending. The results are familiar, but still worth re-reading: Americans generally want to cut government spending, but do not want to cut spending on specific programs; indeed, most people want to spend more on most programs.

Blumenthal brings in Democratic pollster Mark Mellman, who tries to square the circle by noting that most people also believe that large portions of federal spending are simply wasted. Now, that doesn't quite work, at least not entirely; unless people actually believe there's an account marked "Waste" they still support increases in specific programs even though (presumably) they believe that those programs waste money. In other words, I don't think you can really interpret the poll findings without attributing quite a bit of inconsistency to respondents.

Still, "waste" is an interesting topic. Indeed: Bobby Jindal's column today about sequestration was based, more than anything, on incredulity that Barack Obama couldn't find enough waste to cut that sequestration could be implemented "without jeopardizing critical services." Granted, he has no actual suggestions about waste; his only specific suggestion for replacing sequestration is to delay ACA implementation of exchanges and Medicaid expansion. Presumably, that's not "waste" -- whether or not one supports Medicaid for people just over the poverty line and subsidized health care insurance for those making more than that, it's a fairly efficient expenditure in terms of buying what the government intends.

Of course, the question here is what counts as waste. And here's where we could use some help. As far as I know, we have questions about waste, or waste-fraud-abuse, but no good questions about what they means to voters. To me, "waste" would mean, for example, that it takes six government workers to do something that one private sector worker would do. Or: waste would be a government agency buying, say, lots of extra computers which then sit in boxes because they weren't actually needed. Or what Al Gore used to talk about: buying office supplies for ten times what they would cost at Staples because of bureaucratically-mandated procedures. On the other hand, money spent on some government program I don't like -- say, building a wall on the Mexican border -- wouldn't count to me as "waste" in this context, as long as it was done efficiently. But I certainly could imagine someone thinking that such money (or enforcement of drug laws if you think those drugs should be legal, or the Iraq War if you think it was a mistake, or I suppose Social Security if you are against that program) is "government waste." Not only that, but anyone who thinks of it that way is certainly correct that there's plenty of that kind of waste -- but your waste, then, might be my vital program.

That is, there's no "correct" answer about how to interpret all of this. What we need, then, is to learn more about what polling respondents (and, for that matter, politicians) mean when they say that there's tons of government waste. Hey, pollsters! How about some questions to help us understand it. And, hey, reporters! How about pressing politicians when they claim there's lots of waste.

C'mon, Folks, Polling Averages. Remember?

Ron Fournier on one reason Barack Obama may refocus his State of the Union message:

Though Obama's team would dismiss its significance, Democratic allies took notice of Quinnipiac University's new poll that showed Obama's approval rating dropping since his election, from a 53 percent approval rating in December to 46 percent.

C'mon. Did we all learn nothing from the last election cycle? Polling averages, folks, not single polls. Heading over to Pollster, guess what? Obama's approval rating, including that Q poll, is rising, not falling. In fact, there were a total of polls with surveys ending February 4: the approval ratings on those were 46, 46, 52, and 54. And the three polls after that Q poll? Approval readings of 49, 52, 52. Conclusion: the Q poll was a bit of an outlier, or at least it was at the low range of the average.

(Even then, 46% approval is misleading; the two 46% polls both had high undecideds, so Quinnipiac had Obama with a net 1% positive approval score, and YouGov had him dead even. That's worse than Gallup and the others, but not as much as looking at just the approval would suggest).

Polling averages are not just for election season. Be wary -- very wary, on issue polling -- of single polls on any subject, at any time. I don't know whether the fault here is with Fournier or his sources, but everyone needs to learn this very, very, simple lesson.

Elsewhere: Filibuster, Silly Polls, More

Sorry for the slow blogging today; we had a power outage here at Plain Blog World Headquarters this morning (on top of a messed up night last night, as those who noted the time stamp on the Watergate post might have noticed), and next thing you know I hadn't actually wound up with anything to put here.

I'll try to get back later with a baseball post...we'll see. Meanwhile, I wrote an Ignore Those Polls! post about the House Republicans and issue polling on the debt limit over at Greg's place. That episode reminded me of Newt and Luntz in 1994/1995, but yesterday I suggested that perhaps, following that model, the 113th Congress might be not as awful as the 112th.

Meanwhile, at PP, I argued today that there really is a potentially solid compromise on filibuster reform.Yes, Superbill! gets a mention.

Ignore Those Polls!

Well, actually, sort of the opposite -- two great polling questions today furnished an excellent reminder of why all policy surveys should be taken with a monstrously large grain of salt.

First, one poll has 47% believing that going over the "fiscal cliff" will increase the deficit -- while in fact, the combination of tax increases and spending cuts will radically decrease the deficit.

Second, as Sarah Kliff reports, PPP invented and polled about an entirely fictional "Panetta-Burns" plan, and found that about a quarter of respondents were willing to state an opinion about it.

The point isn't that voters are stupid, which I don't believe, or even that they're just not very informed, which I do believe. The point is that most of us don't bother to develop real opinions about lots and lots of things, but yet are nevertheless willing to answer pollsters' questions.

Look, I'll give an example from something removed from politics. As regular readers know, not only am I a pretty serious baseball fan, but I write about it here all the time (and more; long ago, I published a few things in Baseball Prospectus, so yeah, I put in time and energy on this stuff). Indeed: every year I write a post about the Hall of Fame ballot, and I take those posts pretty seriously, doing enough research that I'm satisfied with the results. I've also read, thought, and argued quite a bit, about the Hall of Fame itself, and about virtually every player who comes up for a vote. And yet: since I'm still a couple of weeks away from my HOF post, I would really have to refresh my memory to see what I had decided about some of the close cases. And if I received a PPP poll asking me about the HOF ballot this year, I'm sure I would try to answer, but I also think there's a pretty good chance that I'd get one or more of the answers "wrong" -- in the sense that I'd give a different opinion than what I had written last year and will write again this year.

If that's true of me and the HOF ballot, just think how true it would be of the average voter and some issue she rarely thinks about. Of course you're going to get some goofy answers. Of course things such as question wording and other survey treatments can have a huge effect on the results.

Again: it's easy to make fun of the respondents, but that's the wrong reaction to this sort of thing. If all those people who got the fiscal cliff wrong suddenly were appointed to Congress and had to vote in committee on it tomorrow, they would bother to learn about it -- just as we all wait until we need it to learn all sorts of things in our lives.

So, you know, enjoy this stuff (and, yeah, I really wish someone asked more specifically if the tax increases and spending cuts in the fiscal cliff would increase or decrease the deficit). But just remember not to take public policy polling all that seriously. Yes, there are real opinions that we can get at with careful research, but it's very, very, easy to get junk even if you're trying to ask good questions. And not everyone is doing that.

Those Romney Internal Polls

The New Republic's Noam Scheiber has (some of) Mitt Romney's internal polling from the last weekend of the campaign, and has discussed it with Romney pollster Neil Newhouse.

It's worth a few comments.

First, as Andrew Rudalevige notes, these polls do not justify the type of confidence that Team Romney was reported to have on election day. Romney is close, and closer than the polling averages had it -- but he's still losing. This set of numbers really doesn't inspire a lot more hope than the public polling did; both leave open a real possibility that Romney could win if it turns out he runs a bit better than the polls, but neither really give any reason to expect that.

Second: it's very hard to see any harm in it. Every campaign needs to act as though it believes it will win. But that means that every person in every campaign needs to act that way, and the easiest way to do so is to really believe it. I see nothing in this story, and generally I've seen nothing in any of the post-election coverage, to indicate that Romney's campaign erred in any way based on their apparent wishful thinking about the polls.

Third, in particular it's very hard to argue that redeploying resources to Pennsylvania at the last minute was a mistake. It's probably easier to argue that they should have been targeting Pennsylvania all along, but then again I'm not really aware of evidence that they made any significant targeting mistakes, either.

Fourth, it's very much worth tracking the consistency between what is now reported as the inside Romney campaign view and what (many, most) GOP-aligned news outlets were saying.

Fifth, none of this really gets to the key questions going forward, which as I see it are about (1) why Barack Obama's national vote wound up pretty high in the plausible range of what the fundamentals predicted, and (2) whether the Democrats have opened up a long-term electoral college advantage.

Now, I think all of this is based on the premise that there really wasn't much that Romney could have done about most of the lay of the land. Could he have discouraged African American and young voters from coming to the polls? Hard to see that. Could he have flipped the Latino vote? No, not really; on both policy and, really, rhetoric, his hands were tied by his party.

What he had the ability to do, especially in the final weeks of the campaign, was to move around resources to where they would be most useful. As far as I can tell, Team Romney mostly did a reasonable job of that. They also had control of their own turnout operation, which received lousy reviews...but it's pretty hard, I think, to tie that to polling-based overconfidence, or really anything beyond managerial incompetence (assuming that is that it's true, which we don't really know). So while there's plenty of interesting material in this scoop, I'm not sure how much it really tells us at the end of the day.

Another Note on Exit Polls

I do think the way CNN (and presumably other networks, but I was mostly watching CNN and don't really know what others were doing) are handling the exit polls is a lot better than the way they used to do it. On CNN at least, as the polls closed in each state, they either called it for a candidate, or showed the exit poll head-to-head estimate.

For one thing, those later exit poll estimates are presumably (I haven't checked) more accurate than the earlier ("leaked") ones. For another, the game where they know and are reporting on those polls while keeping the top-line estimates secret was always both silly and annoying. Because everyone knew that the TV people had information that viewers did not have, it pushed everyone to interpret anything said on TV through that secret knowledge. Which also encouraged everyone to believe, falsely in many cases, that the TV people knew exactly what had happened.

However, what they really should do is to report them as ranges, or confidence intervals, rather than as specific point estimates. Put a good graphic up and I don't really think it would be too confusing for most viewers, and it would be a lot more accurate than what they're giving us. And it works for the networks, too: by emphasizing uncertainty, it presumably makes it more likely that people will stick around. After all, the real idea here is that we the viewers should know what the people on TV know. Giving the specific estimates doesn't really do that; it implies a kind of precision that they don't really believe -- or else they would have called the state already!

Granted, this doesn't solve the problem of keeping the exit polls secret before voting ends in each state. But that's most easily solved by not turning on the TV until Indiana and Kentucky close, or even better not until an hour after that when real returns start coming in. By then, we'll have enough information (via the early calls and the exit polls in the other early-closing states) to know what they know, even if they have aren't going to reveal the exit polls from late-closing states for a while.

Leaked Exit Polls, Revisited

Remember back on Monday and Tuesday when this blog and several others were cautioning everyone to ignore leaked early exit polls?

I figured I should go back and check to see whether that advise turned out to be sound or not. Quick answer: yes. Ignoring the exit polls and using the pre-election polls gave a much better indication of what was happening.

One thing that's changed is that we no longer get early exits leaked as early as we used to. In the old days, insiders would hear about exit polls in midday, reflecting the earliest wave of polling. Now, the networks really don't allow anything to leak until much later; of course, when it does leak, everyone with a computer hears about it.

So when they did leak in late afternoon or early evening (and Taegan Goddard's wonderful Political Wire reported them), how useful were they?

The leak, as Goddard reported, included 11 states.

First cut: compared to nothing at all, the leaked exit polls (LEPs) did okay. They reported leads in 9 states, and only had one wrong: the LEPs had Romney up one in Florida, but Obama won it by 0.6. The LEPs were within three points in all but two states. That seems at least somewhat useful. However, there was also a solid pro-Romney bias in the LEPs; In nine of the eleven states, the LEPs had numbers better for Romney than the actual results.

Second cut: Add all that up, and the LEPs gave the mistaken impression of a narrow and not certain Obama win. With Florida leaning Romney and Virginia and Colorado tied, the LEPs had the election Obama 281, Romney 235, undecided 22. With Ohio one of the two states where the LEPs favored Obama (showing a four point lead, while it wound up only two), there was no obvious way for Romney to get the rest of the way, but a whole bunch of targets: he was losing in the LEPs by three in Minnesota, Iowa, and New Hampshire, and by four in Ohio, Pennsylvania, Wisconsin, and Minnesota. For what it's worth, there were no LEPs, at least in this set, for Michigan, New Mexico, or Oregon. So what you're left with if you were hoping for Romney was that the supposedly tied states would go his way, and that somehow he would find another 13 EVs. Unlikely, but perhaps slightly more likely than what the polling aggregators and predictors had been saying.

Third cut: but that's when we get to why these LEPs were actually totally useless. Going back to HuffPollster's final polling averages, the LEPs beat pre-election polls in only New Hampshire and Nevada, and New Hampshire only by a bit (Obama's lead was 3 in LEP, 2.5 in HuffPollster; the vote was a 6 point win). In three other states, LEP and HuffPollster were tied. So in six of eleven states, using the pre-election polls was better than using the leaked exit polls. Or, if we want to look at the simple average miss, the LEPs averaged miss was 2.8, while HuffPollster in these states averaged 2.2.

So, yeah, remember it for next time: ignore leaked early exit polls.