Lessons for Pollsters in the Election

Val Smith, PhD, Wayne Johnson, and Bereket Kelile
Val Smith PhD is Research Director for California based SJR Opinion Research with 35 years of State, National, and International survey research experience. Wayne Johnson is the founder and Senior Advisor to SJR Opinion Research and Bereket Kelile is Associate Director of Research.

When it comes to elections, just like sports, the spread does matter. Democratic strategist James Carville said the day before Election Day on MSNBC that “we’re going to know the winner of this election by 10 tomorrow night.”

The big surprise on Tuesday was that the race was much more competitive than we were led to believe. Several polls in the weeks ahead of the election put Biden’s lead in the high single-digits to low double-digits. An ABC/Washington Post poll showed Biden leading Trump in Wisconsin by 17 points only days before the election. Biden won Wisconsin by a razor-thin margin. The media polls never showed Maine Senator Susan Collins with a lead at any time in 2020. She won anyway.

 In fact, the much-anticipated repudiation of Trump and Trumpism was nowhere to be found, while the predicted blue wave was just a calm ripple.  While the blame will be heaped on pollsters, the real culprit may well be their media clients who dismissed any poll that didn’t fit their narrative.

Most of the chatter will focus on issues like the “shy Trump voter” effect and bad sampling techniques, i.e., not correctly identifying who will likely vote. But there may be a deeper problem. It’s called confirmation bias, a propensity to believe information that confirms what you already believe. Pollsters use it to describe behaviors of those they poll, only in this case, it may be describing the pollsters and their clients.

The large national polls are often commissioned by the media (NBC News/Wall Street Journal; Washington Post-ABC News; Fox News Poll), which presents two possible problems. National media giants made no secret of their hostility to one of the candidates, so much so that only poll results  that conformed to their bias won the coveted “respected poll” designation, while those that did not conform to that bias were dismissed as “outliers”.

Robert Cahaly, of the Trafalgar Group, was criticized for claiming that media polling was underestimating Trump and down-ballot Republicans in Senate races. Nate Silver, of FiveThirtyEight, said that a Trafalgar poll in Michigan was “just crazy.” Josh Pasek, a professor at the University of Michigan, took issue with Cahaly not revealing his methodology: “If somebody’s not transparent you can generally assume they’re crap.”

These outlier polls were not right because they couldn’t be. There simply could not be that many “deplorables.” And yet, as the dust settles, it appears these “outlier citizens” voted for a Republican Senate and gave Republicans the victory in 25 of the 27 House races the media had identified as “toss-ups”.  Even more inexplicable to those living inside the confirmation bias bubble, a lot of those outlier voters were Latinos.

Smart Democrats are already questioning whether these information blind spots prevented them from seeing how much the specter of socialism, civil unrest and cancel culture may have hurt the party.

Republicans sifting through the same post-election rubble have the tougher job, separating the messages from the messenger and crafting an agenda that can survive and flourish in a post-Trump world.

The great thing about elections is that they keep pollsters honest because the results serve as the final grade by which a methodology will be judged. As Warren Buffett said, “when the tide goes out you know who’s been skinny-dipping.”

Comment on this article


Please note, statements and opinions expressed on the Fox&Hounds Blog are solely those of their respective authors and may not represent the views of Fox&Hounds Daily or its employees thereof. Fox&Hounds Daily is not responsible for the accuracy of any of the information supplied by the site's bloggers.