I don’t pretend to understand a huge amount about the science of opinion polling – like any political geek, I dabble, but that’s it. However, I do understand a little more of the art of opinion polling, of manufacturing the response you want to deliver a story.
So I was pretty disappointed to read the latest ComRes poll, commissioned by The Independent, which asked a couple of deeply flawed questions. Here’s the worst of the bunch, asking the public to agree or disagree with the following statement:
The political horsetrading which followed the inconclusive General Election result showed that an outright win is much more desirable than a hung parliament.
Now it’s an interesting question to find out what the public did feel about Britain’s first taste of post-election bargaining in the full glare of rolling news – I’d love to see a genuine question putting it to the test.
But this question by ComRes is risible – a serious pollster would not use the loaded term ‘political horsetrading’ if they were genuinely interested in finding out the public’s views. Unsurprisingly the public reacted negatively to the notion of ‘horsetrading’, with 74% saying they preferred an outright win.
What, though, if the pollster had tested the public’s attitudes with a more neutral question, something like:
On 7th May no single party won an overall majority, so the three major parties opened discussions to see if they could reach agreement. Some people say that this is a good thing, resulting in a programme for government based on the most popular policies from each of the parties. Other people say it is a bad idea, and that it would be better if a single party had won an outright majority and could govern alone. What is your view?
Now I wonder what might have happened if ComRes had asked that question. I suspect they would have got a very different answer to the one they got. Which leaves one question hanging … Why did they ask a slanted question that they must have known would provoke a negative response?
Voting intentions:
For the record, ComRes found support for the parties as follows:
- Conservative 37% (-1), Labour 33% (-1), Lib Dems 21% (n/c)
This is consistent with most other polls – indeed, I think it’s the sixcth consecutive poll which has shown the Lib Dems at 21%. Okay, the balloon of ‘Cleggmania’ may have popped, but that’s a lot better than some of us would have dared hope in the immediate aftermath of the coalition deal being struck. But it’s very early days of course.
ComRes also tested post-coalition attitudes to the Lib Dems, using one of those “putting words in your mouth” questions beloved of newspapers but which brings political science into disrepute: “Now that they have joined a coalition with the Conservatives, it is difficult to know what the Liberal Democrats stand for” – 65% agreed, but 29% disagreed. Given it’s a statement which invites the public to offer a nod of cynical agreement, I was mildly encouraged that getting on for one-third of the public disputed it.
Electoral reform:
ComRes also asked a question about changing the voting system, testing the statement: The first-past-the-post system for elections to the House of Commons should be replaced by a system that reflects more accurately the proportion of votes cast for each party.
It’s welcome, though entirely unsurprising, to discover that 78% agree, with only 18% disagreeing. It would have been more interesting, though, to find out what the public thought of the Alternative Vote (a non-proportional system) compared with a genuine system of proportional representation (such as STV) or the status quo of First-Past-The-Post.
A propos almost of nothing, here’s a famous clip from Yes, Prime Minister which looks behind the opinion polls:
Follow @LibDemVoice on Twitter
Like us on Facebook
Subscribe to our feed
Sign-up for our daily email digest
The Sun story about Chris Huhne that was a total invention (and which the other papers happily copied)
11 Liberal Democrat MPs vote for registrars to be exempt from marrying same sex couples
How Liberal Democrat MPs voted on same sex marriage amendments
Same sex Marriage: another Liberal Democrat victory
Opinion: Why Lib Dems should support the Referendum on the EU – and let’s have it before 2015
Opinion: Referendum debate is a sideshow: the countdown to Britain’s real EU vote starts today
The Independent View: An invitation to ORGCon 2013: the UK’s leading digital rights conference
Baroness Judith Jolly writes: new Social Care Bill focuses on people not systems
Opinion polls yadda yadda. OR “Does Nate Silver mean nothing to you? Did he write in vain?”
Friends of Magdalen Green
Welsh Government more interested in fighting UK counterparts than helping people
We can be 'listened at' but not 'listened to'...Mathew Hulbert writes for The Guardian!
Just how open is greater manchester's police and crime commissioner?
Silence of the lambs and howling of the wolves
Woolwhich: The counter narrative
Ealing Council Finds Councillor Dennehy Guilty as Charged For Offensive Blog
Tope's Hopes
The Beach Boys Live: The 50th Anniversary Tour
Eddie Sammon
Paul Pettinger
Martin
A Social Liberal
10 Comments
It’s called a leading question. It is well established that they significantly change how people respond. The pollsters all know this – but pollsters are in the business of delivering the answers that their clients asked for (why else would they get paid?).
The electoral reform question is no different – in fact it’s the worst form of these. Almost everybody says “yes” to a “do you want more pie?” question.
It would be lovely to have an independent pollster who was really dedicated to finding out the truth in these matters, but I don’t see how to fund it.
That was precisely what I thought when I read that in the newspaper.
The questions were absurd and I am glad you pointed them out.
The question about what the party stands for ought to be made of the other parties as well by comparison. However at this moment in time, before the policies have even been put in place, and before Labour elects a new leader, then it is hard to tell what anyone stands for, so the question is rather meaningless.
So 78% of people want a more preportional voting system but 74% want a party to have an out right win. Surely the conlusion is that most people polled by Com Res don’t have a logical mind?
Surely everyone knows what a leading question is?
That’s how it is done
The best ever was the News of the world that ran a poll (in the light of Megan’s law) that asked “do you think that Paedo’s should be able to lurk in the shadows”?
The vote was 90% but one must wonder about the 10% who thought they should “Lurk in the Shadows”.
Private eye once ran a spoof Daily Mail headline “Kinnock in child sex poll shock”- with the story (the small print) being that if it was found that if this was the case then his poll ratings would slump.
However, on a more serious side George Bush speeches before the invasion simply had “Al Queda & Saddam” in the same paragraph although there was never any connection made between the two in speeches.
The introduction of that simple rhetorical device saw polling figures that reported a majority believed that the two were directly linked. to each other.
You first define the playing field (the fact that Saddam and Al quaeda were natural enemies did not even enter the debate), then just blow the way you want the discourse to run.
Next local elections, tactical voting out, protest only in one camp as Lib-dem & tory the same (easy to do in a media age that personalises stories -first NHS death linked to a cut should do it.)- The playing field is created.
Then words like betrayal put into the literature and we have got the wind direction. No sophisticated arguement or defence of duty will cut it. You don’t meet logic when you are knocking on doors, you meet mood and emotion.
The mistake that is made is you are looking at the question in the poll. It has nothing to do with the question. It is not biased or leading ; it is emotionally loading the question that is the trick.
Ask the Daily mail.
Maybe it’s because Andrew hawkins, COMRES’s Chairman is a staunch Tory?
@Simon, I can see why you are saying that, but I think all it proves is that you can get an unreliable poll result according to the way you ask the question, as Stephen has pointed out in the article.
The Comres question quoted is an example of ‘begging the question: i.e. embedding an assumption within a question to influence the answer — or, put another way, in which the proposition to be proved is assumed implicitly or explicitly in the premise. e.g. “Have you stopped beating your wife?” Unfortunately, the term “to beg the question” is often used incorrectly as a synonym for “to raise the question” or “to pose the question” often by T.V. and radio journalists. e.g., ‘This year’s budget deficit is half a trillion dollars. This begs the question how are we going to balance the budget?’Obviously in this example “raises” or “poses” should be substituted for “begs”.
Authentically begged questions are often the tools of political charlatans and the more subtly begged questions can easily find ther way into referenda to achieve the desired result. So you Lib Dems should beware of the Tories insisting that they exclusively devise the referendum question on P.R. (If they ever get round to it)
correction ‘their way’.
As an English graduate and former English teacher I have to inform you that the sentence you are discussing is a statement, not a question at all.
@ Judith Brooksbank
As an M.A. and also a former English teacher I would suggest that context is key here. A question requires an answer: i.e. a binary response. A statement doesn’t necessarily require an answer. A statement made in an opinion survey context and which requires a binary response must therefore be a question. You either agree or disagree with the statement. The same statement made in a different context and not requiring a binary response would be, as you suggest, purely a statement. The Comres example is therefore contextually a question and it is begged even though it masquerades as a statement. Further evidence of how subtly begged questions can be interpolated into surveys and referenda.