Subscribe to DSC Newsletter

3 Post Election Lessons About Polling & Behavioral Data

The entire polling industry faced an existential crisis on the grim morning of November 9, 2016. The morning before, on Election Day, nearly every mainstream media outlet, from the  New York Times to the Princeton Election Consortium, predicted Mrs. Clinton’s change of winning from 70 to 99 percent. Democrats went to the polls with a sigh of relief. Cakes were baked, firecrackers were bought, and media sites tailored their content for a Hillary win.


Then Trump won in an Electoral College landslide. The polls had failed miserably, but this wasn’t the first time in 2016 that they had confidently predicted the exact opposite of the result. Just a few months earlier, England had voted to Brexit, against all odds and polls. The industry should have taken stock then.


“It’s the overselling of precision,”  said Dr. Pradeep Mutalik, a scientist at the Yale Center for Medical Informatics, who had predicted that some of the polls could be off by 15 to 20 percent prior to the election. If polling is done correctly, using a true representative sample of the population, then the results should always be within the margin of error. But reputable sources had placed Mr. Trump below that margin —a sign that the poll’s fundamental assumptions were wrong.


It will take months, even years, for the industry to re-align and come up with an accurate model for the 21st century—if, in the age where young people are more likely to express opinions on Twitter than sit through a ten-minute phone call with a random pollster, the “polling industry” as we know it can even survive. It would be smart for them to take heed of these three lessons.

1. Watch for confirmation bias

The media isn’t totally dominated by liberals, but it is dominated by elites, who have watched their handpicked presidential candidates, on the left and right, ascend the presidency since World War II. The party nominating processes are traditionally a time for corporations, party elites, and others to whittle out the populists, revolutionaries, and true progressives—for example, Bernie Sanders this year. The Republican establishment would have done the same to Trump if didn’t have his own deep coffers to draw from.


The media, which had painted (for not unjustifiable reasons) Trump as a sexist, racist, buffoon, could not fathom a world or America in which he became President. To do so would challenge their basic assumptions about America, the world we live in, and humanity itself. Their subconscious desires affected their presentation of the data.


After all, there were signs on the morning of the Election that Clinton was in serious trouble in the Midwest. But the Democratic Party ignored those in favor of the outcome it expected—that the Midwest, for decades a bastion of working-class Democratic voters and a part of the Obama coalition, would stay that way. They were caught by surprise because of their confirmation bias—the tendency to interpret new evidence as confirmation of one’s existing beliefs and theories.


Computers can collect and analyze data, but its up to humans to interpret it. If the pollsters and media, many of whom are based on the coasts, away from the America that voted for Trump—they would have seen that a 1-2% percent lead in states Democrats traditionally won by landslides was no lead to feel good about, and rather a cause for extreme concern that was never addressed. It was Trump who spent the last hours of the campaign in Michigan, a state he won by 10,704 votes.


2. Current analysis tools are outdated

Most pollsters currently use the same methods of decades ago—calling phones, assembling focus groups, etc. Clearly in retrospect, there are certain groups that those methods don’t reach. Social media is this generation’s media—and so “social listening”  techniques on sites like Reddit and Twitter can serve as a more effective gauge of a population’s mood.


Additionally, a complete transition out of landlines is needed (landlines are heavily skewed towards older people), as well as movements into web and digital polling, supplemented by creative programming to engage internet denizens.


3. Prepare for the human factor

What won the election wasn’t the the rabid Trump fan base or the Clinton coalition. What gave Trump the victory were the last few percentage points of undecided voters, who broke at the last second largely in favor of Trump. In a change election, more undecided voters will, at the last second, vote in favor of change, which was Trump.


How the polling industry incorporates this is important. The world may seem easy to figure out with enough big data. But there is always a human element that, well, only humans can account for.

The human element is what enabled Dilbert author and hypnotist Scott Adams to correctly call the election for Trump a year in advance. Regardless of how he was doing in the polls, Adams always factored in the persuasion element. Pollsters can only go so far when given an “undecided” response. It is psychological insight that predicts where the undecideds go, and that predicts election.

Views: 862


You need to be a member of Data Science Central to add comments!

Join Data Science Central

Comment by Mirio De Rosa on May 8, 2017 at 6:47am

Actually, the Trump-Clinton polls were not wrong. They were interpreted in a faulty manner. The media circuit was responsible for having broadcasted inaccurate information, and I tend to believe it was a matter of poor technical education (how to read poll results).

Read this and you get the whole story.

Cheers, Mirio

Comment by Vic H on May 6, 2017 at 4:23am

Check out this article on "Non-sampling error" relevant to this discussion:

Comment by Terry Kaufman on May 4, 2017 at 1:00pm

"the “polling industry” as we know it can even survive."  The polling industry as we know it died a slow death sometime between 1995 and whenever.  

"It is psychological insight that predicts where the undecideds go, and that predicts election."  I disagree that "psychological insight" would be an adequate reincarnation or replacement for polling. Scott Adams merely guessed correctly a year in advance, as did I very early in 2016, well before the primaries and caucuses.  All I said, however, is that "Trump could beat Hillary."  I had never heard of hypnotist Scott Adams, but I'm fairly certain that he and I were not the only two people to "correctly call" the election many months ahead.

Nor do I think that pollsters oversold precision. Precision was effectively, if not always overtly, simply ignored.

"...To do so would challenge their basic assumptions about America, the world we live in, and humanity itself."  Lisa, I take it you know what those assumptions are.  My guess is that your assumptions and my assumptions about their assumptions differ substantively. "Their subconscious desires affected their presentation of the data." As far as data go, my assumptions is that "the media" don't know data from doo-doo.

Lisa, your blog post as a whole is praiseworthy in its intent, but in my opinion you tried too hard to make the junk called "polls" and media sound worthy of the effort.  Creative (and complete) destruction of political polls is the best hope for developing a new means of measuring opinion.  Prediction should be the province of the obvious, or to the harmless, or left to prophets.


  • Add Videos
  • View All

© 2019   Data Science Central ®   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service