After a notoriously bad 2016 election cycle for pollsters, expressions of doubt about the reliability of traditional polling methods has been circulating.
It is easy to dismiss inaccurate polls with claims that they are bought and paid for by political groups, or that the pollsters ask their questions in such a manner that allows them to get the results they desire. Sometimes this is indeed the case, but with the 2018 midterms on the horizon, I decided to take a look at what is happening behind the scenes in American political polling.
Polling accuracy is intertwined with another huge problem that is seen as a crisis in the polling world, the cost to conduct polls in the first place. Due to a decrease in the use of landline telephones and increased use of Caller ID features, political polls generally result in a dismal 10% response rate, down from 35% in the 1990’s. In turn, smaller polling samples are not always representative of the views of the public, affecting the accuracy of the polls.
Furthermore, when the cost of polling increases, that cost is passed along to the consumer of the polls, which consists mostly of media outlets. In response, media outlets have become more inclined to cut polling budgets. The bottom line is that with so many issues in the polling world, accurate polls are much less likely to reach their intended audience— the American people.
The key to an accurate poll is the size and randomness of the sample. A random sample must adequately account for multiple demographic categories. Though a large number of respondents is preferable if a sample is truly random, as few as 800 respondents can make for a relatively accurate poll in a presidential election. Considering that the American population is roughly 330,000,000, 800 respondents is an astonishing feat. Randomized samples though are often the hiccup.
Consider a traditional polling practice, where the polling group uses an auto-dialer to call potential respondents. Federal regulations require that cell phone numbers be dialed by hand, which either eliminates auto-dialing as an option or drastically increases the cost of the polling group by forcing them to hire individuals to dial phone numbers and conduct surveys one by one. In response to such a choice, the polling group will usually choose the least costly route, auto-dialing landlines only. Considering that nearly 75% of the landlines in America are owned by those aged 65 and older, these polls will automatically be skewed by an age bias. This is just one example of demographic troubles facing pollsters.
With difficulty collecting truly random demographic samples, some polling groups have relied instead on statistical models weighted for demographics. Relying on statistical models has proven to be ineffective, as an incorrect assumption in the model will lead to a completely unreliable poll.
How bad are the polls, exactly?
FiveThirtyEight is one of the most notable names in polling. In an admittedly wacky and nearly-unpredictable 2016 presidential race, let’s have a look at their predictions.
FiveThirtyEight predicted that Hillary Clinton would win with 71% certainty. With 83.5% certainty, they predicted that Clinton would win Wisconsin. Donald Trump won that state by a .7% margin. Similarly, they predicted that Clinton would win the state of Michigan with 71% certainty. Trump won that state by a .3% margin. In Pennsylvania, there were similar failed predictions. In Arizona, a state that FiveThirtyEight predicted Trump would win with 66% certainty, he only won by a 4% margin. Given how close the race was, accurate polling should have reflected a toss up in each state, with each candidate having about a 50% chance to win.
A public opinion poll by The Huffington Post recently showed that 50% of Americans either “don’t trust polling very much” or don’t trust it “at all.” Given these results, whether to trust this poll becomes a deeply philosophical question.
Some experts believe that traditional polling is going the way of the dinosaur. “I wouldn’t be surprised if the polling business as we know it is not around in 2020,” Karlyn Bowman, public opinion expert at the American Enterprise Institute, said.
The recourse for politically interested citizens may come from free-market innovation, as it so often does. For example, OpenPoll, a pre-launch startup, plans to use blockchain technology to conduct “decentralized, verifiable, and anonymous” polling. The platform will allow users to answer polling questions via a mobile application, with monetary incentives for each answer. This eliminates the need for interested parties to hire polling companies at five-figure sums, and allows them instead to send targeted polls directly to demographically diverse (or similar) respondents. It also eliminates the possibility of deliberate manipulation or mischaracterization of polling results.
Polling results regarding the 2018 midterms are already a contentious issue. Though the future of polling remains uncertain, the ability to gather information from the public remains essential in the interest of furthering political discourse.