RIP polling

Polling died last night.

It’s been terminally ill for a while now. The predictions for the UK general election in 2015 were abysmal. Brexit polls were unreliable. And polls put the Scottish independence referendum result in 2014 as a close call when it was a resounding “No”.

After its performance in last night’s US presidential election, we have to reach for the life support switch.

Here are the poll results immediately prior to the election.

FiveThirtyEight, which became famous for its predictions in previous US elections, had Clinton with a 71.4% chance of winning over Trump’s 28.6%. It also had Clinton comfortably winning the popular vote by 48.5% against 44.9%.

The Independent reported on a model from Moody’s Analytics that has correctly picked every president since 1980. Well…had. It forecast Clinton would pick up 332 Electoral College seats to Trump’s 206.

Sam Wang of the Princeton Election Consortium called the result on 18 October—a win for Clinton. He tweeted

It is totally over. If Trump wins more than 240 electoral votes, I will eat a bug.

That’s lunch today sorted then.

So much time, energy and column inches have been spent on techniques which, again and again, have come up short. We can still have these polls, but we need to get them off the front page and make space for them under the horoscopes.

The problem is that we all desperately want to know what is going to happen. So, there’s a market for people who profess to be able to tell us. Given the demand, if we are going to euthanize polling, we need a replacement.

Prediction markets seemed promising. However, when I looked at Betfair Predicts before the election it was giving Clinton an 83% chance of success. An average of nine predication markets (including Betfair) published just before the election gave Clinton a 82.5% chance of victory.

So much for that then.

Polling (and betting) is based on obtaining people’s opinions—ideally a lot of people’s opinions. Unfortunately, when we lack any reasonable precedent for a situation or decision, it’s very hard to have any kind of informed opinion. Becoming informed about complex socio-economic situations takes resources—an investment very few are willing to make just to enhance the accuracy of a one-off prediction of an event they can’t change.

Crowds have no wisdom when the individuals don’t have a clue.

What about those who put a bit more time into their opinions? Well, the superforcasters at the Good Judgement Project reported a 76% chance that a Democrat would win and a 64% chance that the Democrats would control the Senate.

So much for that then.

Is there anything we can do to predict the outcomes of these elections?

As physicist and Nobel laureate Neil Bohr said

Prediction is very difficult, especially about the future.

If we can’t rely on judgement the only way forward would seem to be to improve our ability to model and study the social physics of complex systems—such as the research published in the Journal of Artificial Societies and Simulation. We’re currently a long way from being able to use such approaches with any degree of confidence, but the techniques used in this field, such as agent-based simulation, have the potential to make predictions in novel situations.

Such techniques also tend to be expensive to use—especially when compared with running an online survey. However, we can’t just keep doing things that clearly aren’t working just because we can.

RIP polling. I won’t mourn you.