Polls apart

Political dimension

Will Jennings, Professor of Political Science and Public Policy and Director of the Centre for Citizenship, Globalisation and Governance at Southampton, was also part of the expert panel on the inquiry. Will is a political scientist who has studied the relationships between polls and the final election outcomes, in the UK and in many other countries. During the 2015 general election in the UK, Will and his team created a prominent forecasting model, featured on the New Statesman’s website, that took both historical and current poll data into account to give a probabilistic model that predicted the swing from the last election and therefore who would win each constituency.

“If you look over time at how opinion polls have performed since the 1940s, they were actually more predictive historically than they are now. This arguably reflects the nature of modern electorates, which is that voters are more volatile. They’re less aligned to social and economic groups, which traditionally meant that if you were from a particular background you were more likely to vote in a particular way,” Will explains. “This made pollsters’ jobs much easier.”

Today, there are many more polls carried out because of increased competitiveness in the polling industry and the reduced costs of internet polling. “Nearly 2,000 polls were carried out in the last election cycle alone, whereas from 1945 to 2010, there were only around 4,000 polls; so around a third of all polls ever conducted were carried out in the last five years. There might be an argument around pollsters having to think about new innovations to improve their methods, or returning to conducting fewer randomly sampled face-to-face surveys, which are more expensive but more reliable.”

Looking at historical polling data is also useful for forecasting because political parties tend to have underlying levels of support. “If the polls tell you a party is below its historical equilibrium, there’s a tendency for the party to gain in support in the final weeks; conversely, if it’s significantly above its historical equilibrium, there’s a tendency for it to drop,” says Will. “But this historical pattern didn’t materialise in 2015, and the polls didn’t move much during the campaign.”

Historical opinion poll results also help researchers understand systematic biases; a fundamental problem with the polls leading up to the 2015 general election was that the polled population was slightly different from the voting population.

“This has historically been one of the biggest challenges for the polling industry: the sorts of people who are responding to phone and internet polls are not representative of the general population. And you really see this amplified if you look at particular subsets of the poll data,” says Will. “The error was slightly bigger in 2015, and also it mattered a lot more because the polls predicted such a close finish, whereas in 1997 for example, where there were also polling errors, it didn’t matter so much in practice because Labour won by such a large margin.”

But how much do the polls themselves influence the election results? “This is very hotly disputed; some people have argued, for example, that the poll results influenced the outcome of the 2015 general election because the expectation of a hung parliament changed the content of the whole campaign, making it about whether the SNP would be ‘holding a minority Labour government to ransom’,” says Will. “However, there is a problem from a research point of view about how to prove this.”

Impact on the EU referendum polls?

With the EU referendum fast approaching, how susceptible will these polls be to the errors encountered in the UK general election last year? Because referenda are far less frequent than general elections, it’s difficult to make historical comparisons. “One thing we do know,” says Will, “is that there tends to be ‘status quo bias’ in referendums: often the polls understate the status quo option because people are generally risk averse.”

Will and his team have also been observing the early polling for the EU referendum. “Interestingly, there are much bigger methodological differences than there were in the general election, and so the telephone and internet polls are telling very different stories,” he adds. “The telephone polls are giving the story that remaining in the EU is the most likely option by a substantial distance, whereas the internet polls have been showing potentially some lead for leaving the EU. This hints at the fact that these samples are very different people. We don't know who’s going to be right until election day, and perhaps the samples will converge as we get towards the election.”

You may also be interested in: