Will Polling ever recover?
I thought about putting this in political philosophy. Anyway...
Most people here are American so I will need to give a background. A week ago may have seen an example of political polling lose credibility for all time; the Australian election had polling for 2 to 3 years that consistently for all published polls predicted an election result. Then the opposite result occurred. The election was won pretty comfortably by the other party! Not only that but here's where it's even weirder; about 2 years ago the Australian Primeminister stepped down because of polling that was against him! So in hindsight he should have stayed the course.
Reasons suggested for this massive total failure of political polling:
[1] People lie and vote differently.
[2] Polling companies are still using outdated methods based upon landlines that ignore social media.
[3] Polling is biased and is run as proxies for political interest paymasters.
Another issue is that polling methods are closed and proprietary. So sampling methods are not known. At most the public is given a margin of error. In my opinion it's the end of commercial polling; I will only from now on look at polls that are opensource where all of the sampling data and methods are in the public domain. But I can't see any hope for the polling industry to recover from this. This was like a world where the laws of thermodynamics could be shown to be a lie. It was seismic.
Can we do better and how? Will you trust any poll anywhere again in any country?
Most people here are American so I will need to give a background. A week ago may have seen an example of political polling lose credibility for all time; the Australian election had polling for 2 to 3 years that consistently for all published polls predicted an election result. Then the opposite result occurred. The election was won pretty comfortably by the other party! Not only that but here's where it's even weirder; about 2 years ago the Australian Primeminister stepped down because of polling that was against him! So in hindsight he should have stayed the course.
Reasons suggested for this massive total failure of political polling:
[1] People lie and vote differently.
[2] Polling companies are still using outdated methods based upon landlines that ignore social media.
[3] Polling is biased and is run as proxies for political interest paymasters.
Another issue is that polling methods are closed and proprietary. So sampling methods are not known. At most the public is given a margin of error. In my opinion it's the end of commercial polling; I will only from now on look at polls that are opensource where all of the sampling data and methods are in the public domain. But I can't see any hope for the polling industry to recover from this. This was like a world where the laws of thermodynamics could be shown to be a lie. It was seismic.
Can we do better and how? Will you trust any poll anywhere again in any country?
Comments (10)
Both polling results and election results were very close:
Without doing an in-depth research, I could only find figures with error margins for Ipsos:
Quoting Ipsos
Their margin of error was 2.3%. Their result was off by 2.5%. "Total failure," really?
I don't see what this has to do with logic, mathematics or philosophy. At most, this is yet more evidence of innumeracy in the general population. By the way, the same hand-wringing accompanied Trump's election victory, where quality polls conducted close to the election were mostly right (i.e. their deviation was within their margin of error).
If someone is consistently winning polling, it could (for just a few examples):
(a) influence people who would vote for the poll winner to relax and not bother showing up to vote (of course in countries where voting is not mandatory), because they figure "the poll winner is going to easily win anyway,"
(b) motivate people to vote who sometimes don't vote and who are supporters of the poll loser, because they think "the poll loser needs all they help they can get--I'd better show up and vote this time," and
(c) sway the vote of people who were on the fence--"I guess I'll give the poll loser a vote, because I'm undecided/I like/dislike both (or multiple) options more or less equally and the poll winner is going to win anyway, so I want to show some support for the loser/show lack of support for the winner."
People make predictions about what's going to happen in elections, and those predictions influence their behavior. Polling helps reinforce predictions, because "It's scientific and all that," so it has a strong influence on folks' behavior. It's similar to what happens with gambling. People make predictions, experts set odds, and that all influences how people bet, where people utilize that info to follow various strategies--they try to "game the system," but that can have relatively unexpected consequences.
So polling can be very accurate while not successfully predicting an outcome.
Polls can be expected to be wrong by a few percentage points sometimes. If they weren't then something strange would really be afoot. There might be some discussion to be had as to why they are wrong in particular instances, but there's nothing particularly strange about the Australian election. It was always predicted to be a close result and it was.
Well, facts and figures are obviously of no help to you - that was my point. When the margin of victory is as thin as that, margins of error will come into play. Really, when you see polls clustered tightly around 50%, even without inquiring after margins of error, you should realize that, as far as predicting the winner, a typical poll is pretty much a crapshoot.
One interesting thing about the election polls cited above is that, while any poll on its own was not wrong (or was not too far off), their combined result was. This is because when multiple polls of similar quality are averaged, their combined error margin is smaller than that of any single poll.
So if we take the average of the latest 5 polls in the table above (discarding the earlier Newspoll result), they give 48.6% for the coalition that ended up taking 51.5% of the vote. Assuming that all the polls had a similar margin of error (2.3% in the case of Ipsos), their combined margin of error is about 1%, which results in a gap of about 2% from the actual result.
Still, like @Baden said, a gap of a couple of percentage points is nothing to be hysterical about. The lesson for you is that when two competitors are going neck and neck like this, nothing short of a time machine is going to give you a very reliable prediction.