back to top
Politics

Polls, Damn Polls, And Statistics

What, exactly, do all these presidential election forecasts do? And with just two weeks left, why are we still clicking them so compulsively?

Posted on

Election forecasts are everywhere.

BuzzFeed News; Getty

It’s late October, which means — if you’re like us — you’ve probably developed a repetitive stress injury from constantly refreshing your favorite election-forecast website. FiveThirtyEight, the RealClearPolitics, New York Times, the Huffington Post, Talking Points Memo and more; every political site is trying to predict the 2016 presidential election.

Tracking them obsessively can feel like a way to cope with our election-related anxiety — though it probably does more to fuel that anxiety than to allay it.

The sites all say they are forecasting the election, but they have different methodologies and, at times, substantially different results.

BuzzFeed News

Most forecasts base their projections largely on polls. Even among that group, however, there are different ways to weigh and adjust the responses and calculate the degree of uncertainty. And that's not the only way to predict an election. PredictWise relies not on polls but on betting markets, which can react much faster to big news events — or choose to ignore them completely.

However the different sites come up with these numbers, they all represent the odds that a candidate will win on Nov. 8, right?

BuzzFeed News; Getty

Not quite: they represent the odds that a candidate will win on November 8, given what we know today.

Let’s take Nate Silver’s FiveThirtyEight “polls-only” forecast. Today, it gives Clinton about an 87% chance of winning. That is: If we found ourselves in this situation — these same poll results, this many days before the election — one hundred times, Clinton would be elected president in 87 of those scenarios, and Trump in 13.

One hundred times? But we’re only going to do this once. (Thank goodness!)

BuzzFeed News

Yes, but it’s not the first time that the U.S. has elected a president. No previous election, obviously, has been exactly like this one, but forecasters can use past polling and election results to get a sense of how different situations played out.

But we haven’t had many elections with big-league polling data. Can we be that confident?

BuzzFeed News

It’s true: Silver’s forecast, for example, is “trained” on just 11 election cycles — 1972 through 2012. (Other use even fewer.) But something working in forecasters’ favor is that the U.S. election is actually a combination of lots of smaller, state-level elections. (Hello, Electoral College!) That gives forecasters’ algorithms a healthy chunk of data to crunch.

Why have the forecasts changed so much over the course of the election, even week to week or day to day?

Getty

Well the obvious reason is that people change their minds. More than you might think. In 2012, CNN’s exit polls found that 21% of voters made their decision in October or November. Nine percent of voters decided in the “last few days.” And even a small change in public opinion in a swing state can have a big impact on the overall election. Remember Florida in 2000?

Another reason: Polls only capture a small slice of voters.

BuzzFeed News; Getty

A big national phone poll might only have a couple thousand responses. These respondents aren’t perfectly representative of Americans overall, so pollsters have to find ways to extrapolate the initial results to the overall population of the United States.

But there’s no single, agreed-upon way to do that; last month, The New York Times gave the same raw data to four different pollsters, who each drew wildly different conclusions.

One more reason: $#!% happens.

BuzzFeed News; Getty

This election has already seen a few “October surprises,” and there could still be more to come. Polling data can't predict major scandals, international warfare, Beyoncé dropping an album, or any other event that might sway the election.

That said, the closer we get to the election, the harder it is for the underdog to catch up with the frontrunner, so most predictions will — all other things being equal — get more confident over time.

Still, no matter how the numbers stack up, it’s all just an educated guess until the votes are counted.

John Templon is a data reporter for BuzzFeed News and is based in New York. His secure PGP fingerprint is 2FF6 89D6 9606 812D 5663 C7CE 2DFF BE75 55E5 DF99

Contact John Templon at john.templon@buzzfeed.com.

Jeremy Singer-Vine is the data editor for the BuzzFeed News investigative unit and is based in Washington, D.C. His secure PGP fingerprint is E2B0 63DB 0601 D634 1E9E F9AE 9F24 768F 9B4A EFB0

Contact Jeremy Singer-Vine at jeremy.singer-vine@buzzfeed.com.

Got a confidential tip? Submit it here.