The pointlessness of political opinion polls

Leo Barasi’s piece over on Liberal Conspiracy raises an interesting point about the frequency of political opinion polling in the UK. We now have far more polls than before giving national voting intention figures (this Parliament so far: 878, 2001-5 312 in total, 1987-92 548 in total – to give some examples). But do we have too few?

Due to the vagaries of random sampling, a poll that shows a party’s support going up or down a couple of points doesn’t really show anything. It’s like tossing a coin 10 times and getting 4 heads first time round and 6 the second time round. The coin hasn’t become loaded towards heads; it’s just random variation.

It’s in fact very rare for an opinion poll to show a statistically significant variation in support from the previous poll by that company (and different ‘house effects’ means it isn’t very meaningful to compare polls between companies). That isn’t the fault of the polling companies; rather it’s the reality that overall people’s voting intentions rarely shift that dramatically and quickly to give a statistically significant change, even if the polls are a month apart.

No surprise really given how little attention most people give to political news (and how few people can recall ‘big keynote speeches’ and the like when pollsters ask them). With only small real changes hidden by margins of error, polling is not a precise enough tool to chart the actual changes of support that might be happening.

It means if you’re interested in the national voting figures for parties and how they are changing – and that’s the headline report on nearly all the polls – almost always the report should be “sorry, no idea if anything has happened”.

Instead what we get is far too much excitement placed on changes that are within the sampling error, as if they signify things they do not. (This isn’t a problem specific to politics, by the way. Many economic statistics, for example, are based on survey data and so have similar issues with apparent changes being within the margin of error – yet those are almost never mentioned and all changes treated as if they are significant.)

That makes them all rather pointless. Fun and interesting if used in other ways (such as to look at longer term trends) yet – if you are going to get excited about the latest point up or down – pointless. You might as well go outside, look at the first car number plate you see, and if its first digit is even shout “Cameron’s plummeting in the polls!”, if it’s odd scream “Miliband soars upwards” and if all you can see are tricycles squeal with pleasure at having entered Liberal Democrat heaven.

The ironic solution, of course, is to have far more polls, as with even more polls the trends could be separated out from the noise rather better. Half-way through this Parliament we may already have had more than one and a half times more polls than during the 1987-92 Parliament in total, but given the purposes to which people like putting them, more are needed.

* Mark Pack is Party President and Co-leader of the party. He is editor of Liberal Democrat Newswire.

Read more by or more about .
This entry was posted in Op-eds and Polls.
Advert

12 Comments

  • John Leston 19th Oct '12 - 9:59am

    Would it not be better to spend the same money on fewer polls but with larger sample sizes? The combination of a longer time gap and a smaller margin of error would increase the likelihood of being able accurately to say ‘support for tricycles soars’

  • Nigel Ashton 19th Oct '12 - 11:32am

    Isn’t the reason why there are so many polls because YouGov does a daily tracker poll. I’d rather have less polls with bigger samples, so the results can be broken down by sub-group with some reliability.

  • keep the polls coming! just made the 5000 threshold on yougov, and awaiting yet another £10 voucher from ipsos mori, almost all for saying if there was an election tommorow id vote liberal democrat.

    Well, i also made £4 the other week on populous, for saying Boris would make a better leader than Cameron, funny stuff,

  • paul barker 19th Oct '12 - 2:54pm

    The fundamental problem with Voting Intentention polls (the polls the media report, mostly) is that most people see them as predicting the next election. Officially, the polling firms say they are “just a snapshot” but look at the layout of UK Polling Report – polling average/ labour majority 100+. It lloks like a prediction & if you read the comments its mostly taken as a prediction.
    Its actually the only reason most people take any notice of polls at all & as a prediction polls are always wrong, except for a couple of months before each election.
    Not only are “the polls” consistently wrong they affect the reality they are supposed to be describing, consistently building up labour & others & running down the libdems. Why do think the owners of the sun are shelling out for daily polls if not because they boost their agenda ?

  • @John, to get a smaller Margin of Error requires exponentially more data. Getting a MoE of 1% requires asking around 10 times more people in the UK than required for a MoE of 3%.

  • The rise of online polling is a hugely worrying trend for the reliability of data. Yes, online polling companies can build decent reputations, but the nature of those taking part in said polls is far less demographically representative than traditional phone polling. In particular because online poll companies rely on people actively seeking to join panels, an activity that tends to be associated with the most politically interested members of the public, including a significant number of party members and immediately puts their respondents into such a small group of the electorate that leaves huge doubts about reliability.

  • Polls and the analysis of polling results should be used to inform and motivate political activity, but outside the rarified atmosphere of the professional pollster it gradually and insidiously becomes a substitute for meaningful political action.

    In today’s relativist world, all headlines are used as justification to avoid issues rather than deal with them.

  • John Leston 19th Oct '12 - 6:07pm

    @duncan. Yes, at total population level but as Nigel comments a larger sample means you can get viable MoEs for important segments ….. After all, in a Nat Rep sample of 1000 how many Lib Dems are you going to get.? 100 ? So, if you want to know what they think then your +/- is big

  • The problem is the ridiculous amount of attention that people (including on blogs!) give to mid-term polls. They might as well give weekly tea leaf readings.

  • Mark Argent 20th Oct '12 - 2:52pm

    I agree that the polls don’t mean much — or at least one needs to be wary of drawing conclusions from changes that are not statistically significant. On the other hand, the conclusions people draw do say something about how the parties are being perceived in the media. That is a million miles from what the polls appear to be measuring, but is still relevant, especially in as much as campaigning is a PR exercise. As an example… an apparent drop in the polls could lead to headlines ranging from “Clegg doing well despite the polls” to “Clegg gets well-deserved hammering in polls”. That does mean we need to engage with them, even if the actual numbers are taken with a healthy pinch of salt.

  • Harry Hayfield 22nd Oct '12 - 10:31am

    All you need to know is that ICM are the only pollster to follow. They have had the best election record going back to the 1992 general election (as proven by Political Betting.com).

Post a Comment

Lib Dem Voice welcomes comments from everyone but we ask you to be polite, to be on topic and to be who you say you are. You can read our comments policy in full here. Please respect it and all readers of the site.

If you are a member of the party, you can have the Lib Dem Logo appear next to your comments to show this. You must be registered for our forum and can then login on this public site with the same username and password.

To have your photo next to your comment please signup your email address with Gravatar.

Your email is never published. Required fields are marked *

*
*
Please complete the name of this site, Liberal Democrat ...?

Advert



Recent Comments

    No recent comment found.