Gallup’s screw-up in charts

Great, great piece at the Monkey Cage by Samuel Best on how Gallup got it so wrong with their pre-election polls.   Apparently their likely voter screen gave them a wildly inaccurate view of the electorate.  Plain old common sense should have told them some adjustment was necessary.  Anyway, here’s the key charts:


 Gallup estimated that non-Hispanic whites would comprise 78 percent of voters in 2012.  This would have been the highest proportion of whites in a presidential active electorate since 2000.  Given the secular downward trend in white composition, this Gallup expectation never had a chance.  In 2012 the exit polls indicate that 72 percent of voters were white, 2 points less than 2008 and a full 6 point lower than Gallup’s expectations…

That the outlier poll (Gallup) was the most inaccurate survey in 2012 should not come as a surprise to Monkey Cage readers.  What is perhaps surprising is that Gallup simply would have needed to review basic voter composition trends by key age groups and racial categories to see the obvious sources of error in their assumptions.  One view might praise Gallup for not adjusting their longstanding likely voter formula to better conform to conventional wisdom or even past trends.  However, moving forward, it seems that Gallup’s likely voter formula will continue to systematically underestimate the proportion of young adults and non-white voters.  As non-whites and young voters increased in size as a proportion of the electorate and became more distinctive relative to the overall voting population, underestimating the proportion of non-whites and young adults in the active electorate has become more consequential for Gallup’s overall estimates.

Gallup has a great reputation in polling.  They can handle a bad election.  If they don’t do dramatically better in 2016, though, their reputation is mud.  Also want to emphasize the point that Best makes that if a poll looks like an outlier, and quacks like an outlier… well, you know.

About Steve Greene
Professor of Political Science at NC State

One Response to Gallup’s screw-up in charts

  1. itchy says:

    “One view might praise Gallup for not adjusting their longstanding likely voter formula to better conform to conventional wisdom or even past trends.”

    Unless you don’t care whether your poll is predictive, this is not a rational view. It’s not a beauty contest. It’s not subjective. There are no style points. Your poll either was accurate or not.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: