Leo Barasi’s piece over on Liberal Conspiracy raises an interesting point about the frequency of political opinion polling in the UK. We now have far more polls than before giving national voting intention figures (this Parliament so far: 878, 2001-5 312 in total, 1987-92 548 in total – to give some examples). But do we have too few?
Due to the vagaries of random sampling, a poll that shows a party’s support going up or down a couple of points doesn’t really show anything. It’s like tossing a coin 10 times and getting 4 heads first time round and 6 the second time round. The coin hasn’t become loaded towards heads; it’s just random variation.
It’s in fact very rare for an opinion poll to show a statistically significant variation in support from the previous poll by that company (and different ‘house effects’ means it isn’t very meaningful to compare polls between companies). That isn’t the fault of the polling companies; rather it’s the reality that overall people’s voting intentions rarely shift that dramatically and quickly to give a statistically significant change, even if the polls are a month apart.
No surprise really given how little attention most people give to political news (and how few people can recall ‘big keynote speeches’ and the like when pollsters ask them). With only small real changes hidden by margins of error, polling is not a precise enough tool to chart the actual changes of support that might be happening.
It means if you’re interested in the national voting figures for parties and how they are changing – and that’s the headline report on nearly all the polls – almost always the report should be “sorry, no idea if anything has happened”.
Instead what we get is far too much excitement placed on changes that are within the sampling error, as if they signify things they do not. (This isn’t a problem specific to politics, by the way. Many economic statistics, for example, are based on survey data and so have similar issues with apparent changes being within the margin of error – yet those are almost never mentioned and all changes treated as if they are significant.)
That makes them all rather pointless. Fun and interesting if used in other ways (such as to look at longer term trends) yet – if you are going to get excited about the latest point up or down – pointless. You might as well go outside, look at the first car number plate you see, and if its first digit is even shout “Cameron’s plummeting in the polls!”, if it’s odd scream “Miliband soars upwards” and if all you can see are tricycles squeal with pleasure at having entered Liberal Democrat heaven.
The ironic solution, of course, is to have far more polls, as with even more polls the trends could be separated out from the noise rather better. Half-way through this Parliament we may already have had more than one and a half times more polls than during the 1987-92 Parliament in total, but given the purposes to which people like putting them, more are needed.
* Mark Pack is Party President and is the editor of Liberal Democrat Newswire.
Follow @libdemvoice.org on Bluesky
Like us on Facebook
Subscribe to our feed
Sign-up for our daily email digest
From Norman settlement to a civic square
Northern Ireland debates the future of its railways in 1957
Church Green video
Thinking of standing for election or re-election next year? Helpful questions to consider
Andrew Tampion
Jeff
Nonconformistradical
John Peters
David Raw
12 Comments
Would it not be better to spend the same money on fewer polls but with larger sample sizes? The combination of a longer time gap and a smaller margin of error would increase the likelihood of being able accurately to say ‘support for tricycles soars’
Isn’t the reason why there are so many polls because YouGov does a daily tracker poll. I’d rather have less polls with bigger samples, so the results can be broken down by sub-group with some reliability.
Nigel: YouGov is partly the reason, but not the whole part – there’s also an upsurge of other polling too.
keep the polls coming! just made the 5000 threshold on yougov, and awaiting yet another £10 voucher from ipsos mori, almost all for saying if there was an election tommorow id vote liberal democrat.
Well, i also made £4 the other week on populous, for saying Boris would make a better leader than Cameron, funny stuff,
The fundamental problem with Voting Intentention polls (the polls the media report, mostly) is that most people see them as predicting the next election. Officially, the polling firms say they are “just a snapshot” but look at the layout of UK Polling Report – polling average/ labour majority 100+. It lloks like a prediction & if you read the comments its mostly taken as a prediction.
Its actually the only reason most people take any notice of polls at all & as a prediction polls are always wrong, except for a couple of months before each election.
Not only are “the polls” consistently wrong they affect the reality they are supposed to be describing, consistently building up labour & others & running down the libdems. Why do think the owners of the sun are shelling out for daily polls if not because they boost their agenda ?
@John, to get a smaller Margin of Error requires exponentially more data. Getting a MoE of 1% requires asking around 10 times more people in the UK than required for a MoE of 3%.
The rise of online polling is a hugely worrying trend for the reliability of data. Yes, online polling companies can build decent reputations, but the nature of those taking part in said polls is far less demographically representative than traditional phone polling. In particular because online poll companies rely on people actively seeking to join panels, an activity that tends to be associated with the most politically interested members of the public, including a significant number of party members and immediately puts their respondents into such a small group of the electorate that leaves huge doubts about reliability.
Polls and the analysis of polling results should be used to inform and motivate political activity, but outside the rarified atmosphere of the professional pollster it gradually and insidiously becomes a substitute for meaningful political action.
In today’s relativist world, all headlines are used as justification to avoid issues rather than deal with them.
@duncan. Yes, at total population level but as Nigel comments a larger sample means you can get viable MoEs for important segments ….. After all, in a Nat Rep sample of 1000 how many Lib Dems are you going to get.? 100 ? So, if you want to know what they think then your +/- is big
The problem is the ridiculous amount of attention that people (including on blogs!) give to mid-term polls. They might as well give weekly tea leaf readings.
I agree that the polls don’t mean much — or at least one needs to be wary of drawing conclusions from changes that are not statistically significant. On the other hand, the conclusions people draw do say something about how the parties are being perceived in the media. That is a million miles from what the polls appear to be measuring, but is still relevant, especially in as much as campaigning is a PR exercise. As an example… an apparent drop in the polls could lead to headlines ranging from “Clegg doing well despite the polls” to “Clegg gets well-deserved hammering in polls”. That does mean we need to engage with them, even if the actual numbers are taken with a healthy pinch of salt.
All you need to know is that ICM are the only pollster to follow. They have had the best election record going back to the 1992 general election (as proven by Political Betting.com).