Tuesday, March 13, 2012

Expect Polling Variation 2

Since I wrote yesterday about not overinterpreting a single poll, I figure I should probably say something about the CBS/NYT poll that came out later in the day and which showed a much more dramatic approval rate drop and a much lower current number (41%).

What should we make of it?

Let me make a distinction I've made before, which is between us as observers and the operatives who must act on incomplete information. For observers, the right answer is simple. The truth is we shouldn't pay any attention to individual polls as long as there are polling averages available. The NYT poll may or may not indicate something new going on (and note that it was a five-day survey that began last Wednesday, so take that into consideration too; see also Jamelle Bouie on the poll methodology). If it does, we'll know soon enough. It is worth noting that unlike the WaPo/ABC poll, this one is in fact south of Barack Obama's current approval rating range.

On the other hand, if you're the Obama White House or the campaign...actually, there's nothing much for them to do, either. At best, they can look for hints about what to test, but they would be making a mistake by overreacting to every poll, too, and for the most part if they have strong evidence that they're doing the right things now, then changing course would probably be a mistake. Note that "doing the right things" won't necessarily produce a win. It's possible that the conditions of the election will make winning impossible. All the campaign can do is to maximize it's vote; it can't affect what that maximized total will be.

Meanwhile, I'll note some differences in the NYT and CBS write-ups of the survey. While the Times quite reasonably began with the job approval numbers, which were in fact the biggest news in the survey, Jim Rutenberg and Marjorie Connelly also included the WaPo and Gallup numbers in the 9th paragraph, thus providing needed context (as Brendan Nyhann pointed out). CBS, on the other hand, did not mention those other polls in either their broadcast or text versions. Even worse, CBS emphasized (in both versions) that this was a "new low." That's true in their own poll -- but Gallup's low is 38%, and others have been as low as 36%. That the infrequent NYT/CBS poll didn't happen to be in the field at those points, or happened to get a higher number, doesn't justify emphasizing "new low" -- it's absolutely misleading. Also bad in the CBS version, especially the broadcast story was the focus on gas prices. The truth is that CBS has no idea at all why Obama's approval rating has fallen in this survey, but their report presented it as clearly a function of gas prices. The Times did mention gasoline and other stories in the new last week, but did not make or imply any clear causal connections.

Obviously, there are constraints on broadcast news that don't apply to newspapers (and especially to online versions of those newspapers). And yet CBS's text version is almost as bad as its broadcast report.

I'll end with a question: I very rarely watch broadcast news, and I haven't for years now. Is it always this bad?

6 comments:

  1. I think people need to stop over-reacting to the over-reaction to an individual poll!

    I mean, of course one poll doesn't itself signify a change in public opinion (and I don't think you'll find any serious commentators making that argument). But it's equally obvious that one poll could be a harbinger of things to come. It's only a single data point, but that data point goes into the mix like any other data point.

    Think of it this way. Suppose a hitter has a season average of .400 as of July. Then, out of nowhere, he gets two golden sombreros in two games, going 0-for-8. Of course, this could be just a normal slump. But it could also mean the player has a nagging injury and will have a horrible second half. We won't know until later.

    Until then, if we wanted to project how the rest of the player's season will turn out, it would be foolish to focus exclusively on the average (and to ignore the two bad games). But that doesn't mean we're over-interpreting the meaning of the two bad games. What the two bad games shows is merely that it is possible something more serious than a temporary slump is going on. And it is that new possibility - new because it didn't exist prior to those two bad games - that gets people exercised.

    ReplyDelete
  2. My sense is that TV networks basically always pretend that their poll is the only one in existence.

    ReplyDelete
  3. Hmm. I think JB's useful spin (don't look at one poll) is morphing into standard, don't worry, Mr. Obama is better than you think spoin which isn't useful.

    Three problems:

    1) As Chiat suggests, it isn't morning in America, and people don't like to be told that.

    2) Ass Kaus, suggests, Obama is just terrible when speaking and the more he talks about the economy (and the less the news focuses on the awful Republcians) the worse people fee.

    3) the issue at hand is gas prices, and Obama is really, really bad on this issue. We want our president to work, not tell us, oh, there is nothing I can do. And Obama secretly does want $5 gasoline. It isn't a bad thing, but there are large sections of this county that don't agree with that analysis.

    ReplyDelete

Note: Only a member of this blog may post a comment.