PollWatch: In which ComRes asks the slanted questions, and I ask them why.

I don’t pretend to understand a huge amount about the science of opinion polling – like any political geek, I dabble, but that’s it. However, I do understand a little more of the art of opinion polling, of manufacturing the response you want to deliver a story.

So I was pretty disappointed to read the latest ComRes poll, commissioned by The Independent, which asked a couple of deeply flawed questions. Here’s the worst of the bunch, asking the public to agree or disagree with the following statement:

The political horsetrading which followed the inconclusive General Election result showed that an outright win is much more desirable than a hung parliament.

Now it’s an interesting question to find out what the public did feel about Britain’s first taste of post-election bargaining in the full glare of rolling news – I’d love to see a genuine question putting it to the test.

But this question by ComRes is risible – a serious pollster would not use the loaded term ‘political horsetrading’ if they were genuinely interested in finding out the public’s views. Unsurprisingly the public reacted negatively to the notion of ‘horsetrading’, with 74% saying they preferred an outright win.

What, though, if the pollster had tested the public’s attitudes with a more neutral question, something like:

On 7th May no single party won an overall majority, so the three major parties opened discussions to see if they could reach agreement. Some people say that this is a good thing, resulting in a programme for government based on the most popular policies from each of the parties. Other people say it is a bad idea, and that it would be better if a single party had won an outright majority and could govern alone. What is your view?

Now I wonder what might have happened if ComRes had asked that question. I suspect they would have got a very different answer to the one they got. Which leaves one question hanging … Why did they ask a slanted question that they must have known would provoke a negative response?

Voting intentions:
For the record, ComRes found support for the parties as follows:

    Conservative 37% (-1), Labour 33% (-1), Lib Dems 21% (n/c)

This is consistent with most other polls – indeed, I think it’s the sixcth consecutive poll which has shown the Lib Dems at 21%. Okay, the balloon of ‘Cleggmania’ may have popped, but that’s a lot better than some of us would have dared hope in the immediate aftermath of the coalition deal being struck. But it’s very early days of course.

ComRes also tested post-coalition attitudes to the Lib Dems, using one of those “putting words in your mouth” questions beloved of newspapers but which brings political science into disrepute: “Now that they have joined a coalition with the Conservatives, it is difficult to know what the Liberal Democrats stand for” – 65% agreed, but 29% disagreed. Given it’s a statement which invites the public to offer a nod of cynical agreement, I was mildly encouraged that getting on for one-third of the public disputed it.

Electoral reform:
ComRes also asked a question about changing the voting system, testing the statement: The first-past-the-post system for elections to the House of Commons should be replaced by a system that reflects more accurately the proportion of votes cast for each party.

It’s welcome, though entirely unsurprising, to discover that 78% agree, with only 18% disagreeing. It would have been more interesting, though, to find out what the public thought of the Alternative Vote (a non-proportional system) compared with a genuine system of proportional representation (such as STV) or the status quo of First-Past-The-Post.

A propos almost of nothing, here’s a famous clip from Yes, Prime Minister which looks behind the opinion polls:


(Also available on YouTube here).

Read more by or more about .
This entry was posted in Polls.
Advert

11 Comments

  • Andrew Suffield 3rd Jun '10 - 12:25pm

    one of those “putting words in your mouth” questions

    It’s called a leading question. It is well established that they significantly change how people respond. The pollsters all know this – but pollsters are in the business of delivering the answers that their clients asked for (why else would they get paid?).

    The electoral reform question is no different – in fact it’s the worst form of these. Almost everybody says “yes” to a “do you want more pie?” question.

    It would be lovely to have an independent pollster who was really dedicated to finding out the truth in these matters, but I don’t see how to fund it.

  • So 78% of people want a more preportional voting system but 74% want a party to have an out right win. Surely the conlusion is that most people polled by Com Res don’t have a logical mind?

  • gramsci's eyes 3rd Jun '10 - 2:27pm

    Surely everyone knows what a leading question is?

    That’s how it is done

    The best ever was the News of the world that ran a poll (in the light of Megan’s law) that asked “do you think that Paedo’s should be able to lurk in the shadows”?

    The vote was 90% but one must wonder about the 10% who thought they should “Lurk in the Shadows”.

    Private eye once ran a spoof Daily Mail headline “Kinnock in child sex poll shock”- with the story (the small print) being that if it was found that if this was the case then his poll ratings would slump.

    However, on a more serious side George Bush speeches before the invasion simply had “Al Queda & Saddam” in the same paragraph although there was never any connection made between the two in speeches.

    The introduction of that simple rhetorical device saw polling figures that reported a majority believed that the two were directly linked. to each other.

    You first define the playing field (the fact that Saddam and Al quaeda were natural enemies did not even enter the debate), then just blow the way you want the discourse to run.

    Next local elections, tactical voting out, protest only in one camp as Lib-dem & tory the same (easy to do in a media age that personalises stories -first NHS death linked to a cut should do it.)- The playing field is created.

    Then words like betrayal put into the literature and we have got the wind direction. No sophisticated arguement or defence of duty will cut it. You don’t meet logic when you are knocking on doors, you meet mood and emotion.

    The mistake that is made is you are looking at the question in the poll. It has nothing to do with the question. It is not biased or leading ; it is emotionally loading the question that is the trick.

    Ask the Daily mail.

  • Maybe it’s because Andrew hawkins, COMRES’s Chairman is a staunch Tory?

  • @Simon, I can see why you are saying that, but I think all it proves is that you can get an unreliable poll result according to the way you ask the question, as Stephen has pointed out in the article.

  • The Comres question quoted is an example of ‘begging the question: i.e. embedding an assumption within a question to influence the answer — or, put another way, in which the proposition to be proved is assumed implicitly or explicitly in the premise. e.g. “Have you stopped beating your wife?” Unfortunately, the term “to beg the question” is often used incorrectly as a synonym for “to raise the question” or “to pose the question” often by T.V. and radio journalists. e.g., ‘This year’s budget deficit is half a trillion dollars. This begs the question how are we going to balance the budget?’Obviously in this example “raises” or “poses” should be substituted for “begs”.
    Authentically begged questions are often the tools of political charlatans and the more subtly begged questions can easily find ther way into referenda to achieve the desired result. So you Lib Dems should beware of the Tories insisting that they exclusively devise the referendum question on P.R. (If they ever get round to it)

  • correction ‘their way’.

  • @ Judith Brooksbank

    As an M.A. and also a former English teacher I would suggest that context is key here. A question requires an answer: i.e. a binary response. A statement doesn’t necessarily require an answer. A statement made in an opinion survey context and which requires a binary response must therefore be a question. You either agree or disagree with the statement. The same statement made in a different context and not requiring a binary response would be, as you suggest, purely a statement. The Comres example is therefore contextually a question and it is begged even though it masquerades as a statement. Further evidence of how subtly begged questions can be interpolated into surveys and referenda.

  • Welfare Central 30th Aug '16 - 10:25am

    @ Judith Brooksbank
    The question mark “?” is a dead giveaway, don’t you think?

    I don’t teach …not a graduate 😉

Post a Comment

Lib Dem Voice welcomes comments from everyone but we ask you to be polite, to be on topic and to be who you say you are. You can read our comments policy in full here. Please respect it and all readers of the site.

To have your photo next to your comment please signup your email address with Gravatar.

Your email is never published. Required fields are marked *

*
*
Please complete the name of this site, Liberal Democrat ...?

Advert



Recent Comments

  • Nonconformistradical
    "So Liz Truss’s ideas of cutting NI and tax rates will not add a single penny to their income!" Exactly. I do wonder if people like Liz Truss have the slig...
  • Peter Watson
    @Peter Davies "the reality these days is that high earners tend to marry other high earners" Also, very high earners are often able to divert some of their inc...
  • Rif Winfield
    Sadly I think that some of your contributors don't appreciate the scale of the assault on the least well off in society. Look for example at pensioners - those ...
  • Alex Macfie
    Roland: Nitpicking? Not really, just pointing out the unenforceability of a distance-based restriction for free train travel, The refund-based model is interes...
  • Peter Davies
    I would challenge you to name the wife of a premiership footballer who is not declaring enough income in their own right to use up their tax allowance. This is ...