The Twilight of the Polls? A Review of Trends in Polling Accuracy and the Causes of Polling Misses

Abstract

Polls have had a number of high-profile misses in recent elections. We review the current polling environment, the performance of polls in a historical context, the mechanisms of polling error, and the causes of several recent misses in Britain and the US. Contrary to conventional wisdom, polling errors have been constant over time, although the level of error has always been substantially beyond that implied by stated margins of error. Generally, there is little evidence that voters lying about their vote intention (so-called ‘shy’ voters) is a substantial cause of polling error. Instead, polling errors have most commonly resulted from problems with representative samples and weighting, undecided voters breaking in one direction, and to a lesser extent late swings and turnout models. We conclude with a discussion of future directions for polling both in terms of fixing the problems identified and new approaches to understanding public opinion.

The 2016 US Presidential Elections: What Went Wrong in Pre-Election Polls? Demographics Help to Explain

“…Our conclusion of an existing problem that needs to be examined and solved is in line with the findings of other scholars such as Prosser and Mellon [34], Cowley and Kavanagh [36], Moon [37], and Toff [38]. All agree that current methods need to be re-examined to improve the accuracy of pre-election polls.…”

“…Some acknowledge an existing problem and try to suggest new methods. Prosser and Mellon [34] describe the failure of predicting the results of the recent elections in the UK and the US. The same also with Lauderdale et al [35] who suggest a new method to improve accuracy following those recent elections.…”

Correlation with Time Explains the Relationship between Survey Nonresponse and Mass Polarization

“…More recently, Prosser and Mellon (2018) show a stable gap between reported and actual turnout in British Election Study data despite declining response rates. Additionally, while declining response rates fit well with rising mass polarization in the US, other countries show various trends in mass polarization (e.g.…”

“…those who do answer their phones must agree to take part in the survey. Declining contact rates have been attributed to things that have nothing to do with political interest, such as the increase in the volume of cold-calling and robo-calls, and the adoption of caller ID (Prosser and Mellon, 2018).…”

Quebec 2018: A Failure of the Polls?

“…If only the first criterion is met—that is, there are significant differences between some polls but not all, or most, of them and the vote—we would call it a pollster failure and look for possible house or mode effects. If the first two criteria are met—that is, there is also systematic poll bias (Prosser and Mellon, 2018)—we would call it a polling miss , and we need to look for possible explanations, first in terms of voter behaviour. Finally, a poll failure is a systematic poll discrepancy that cannot be explained by voter behaviour.…”