Sometimes, Go With "I Don't Know"

Seth Masket has an excellent post up responding to Kevin Drum's question about whether there actually something going on that would mean that "the anti-Washington meme deserves to live."  Seth and John Sides have both been writing about this, at different angles, with John emphasizing that almost all incumbents will win but Seth noting that incumbents may still have to work harder to get there.  I tried to synthesize their views a couple of weeks ago.  

I want to write a bit, however, about why we're not really answering Drum's question with a simple, usable answer, something like either "Voters are turning against Washington pols" or "Voters are not turning against Washington pols."  To begin with, it turns out to be really difficult to learn from primary election data.  In general elections, you can quickly add up how Democrats and incumbents did in previous elections, see how they did in one particular year, and then do a little math and you'll know if there was a general anti-Democrat or anti-incumbent vote (yes, it can still be tricky, for lots of reasons, but most of the trickiness is easy to handle statistically).  Note that it's even easier in House elections than it is in Senate elections, since every election cycle presents the same set of 435 House races.

Primaries are a lot more difficult to study.  What's the "normal" margin of victory for a nominee seeking re-election?  Well, you can calculate that, but unlike general elections you're going to have the competitive elections swamped by a whole lot of incumbents who are unopposed.  We can look at how incumbents do when they're challenged...but how many incumbents are challenged in the first place is an important part of the story we're interested in.  And we might want to distinguish between serious challenges and fringe candidates who file but aren't really a threat, but in practice that's going to be tricky to execute.  Indeed, it's doubly tricky if what we're interested in isn't just the incumbency advantage (or disadvantage), but "anti-Washington" or "anti-establishment" themes.  Does a victory by Member of the House Joe Sestack against Arlen Specter count as an "anti-Washington" result?  If we want to only count serious challengers, how can we tell that Joe Miller in Alaska is serious but Daniel Frielich, who took 11% of the vote against Pat Leahy this week, wasn't?  And yet if you count anyone who files as a serious candidate, then you're going to miss an important difference between Leahy's essentially unopposed renomination and the tough challenges faced by John McCain and Blanche Lincoln. 

So, to answer the question(s) that Drum wants answered, we really don't want just the number of incumbents defeated in primaries this year compared to other years.  What we would want, ideally, would be three numbers, each of which could be estimated independently, holding everything else constant: the effect of incumbency per se; the effect of being an experienced pol; and the effect of being the candidate of the "establishment."  (Even worse!  There could be interactive effects!  There might be different effects on the Democratic side than on the Republican side!).  Unfortunately, there's just no way to derive those effects from primary election data (or at least I believe there's no way -- I'll be happy to hear dissents on this from those who are far more methodologically able than I am).

Barring that, we really have to fall back on just not knowing.  Oh, there are things we can say; we can say (with John Sides and others) that most incumbents are gong to win, and we can say (with Seth Masket and others) that it sure seems that those incumbents have had to work harder than they do in some years.  And we can of course make factual statements (three incumbent Senators losing primaries is the highest since 1980, but four incumbent Members of the House losing primaries is nothing unusual at all).  But sometimes, the best thing that a reporter or a pundit (or, for that matter, a political scientist) can do is to realize which things we just don't know.