My former co-editor here at LibDemVoice, Mark Pack, has been taking a detailed look in his latest Liberal Democrat Newswire at those ICM constituency polls commissioned by Lord Oakeshott. Here’s his take on them:
It’s not only the attempted anonymous funding of the polls that’s questionable. So too is the way they were worded. Even reputable pollsters such as ICM given their clients some discretion over question wording, and in this case it was a matter of misleading by omission rather than an outrageously worded question in its own right.
The full data tables for the polls show they tested the popularity of Liberal Democrats MPs and even some of their named constituency opponents, but when it came to the voting intention question gave only the names of parties and not the names of the individuals.
Yet the names of individuals appear on ballot papers and it’s well established in previous constituency polling that Liberal Democrat MPs do better when named in voting intention questions than when they question is just a generic one about parties. Moreover, naming them gives figures that are better predictors of actual election results.
The only sensible reason for missing out candidate names in polls such as the one done in Cambridge, where the poll tested the reputation of Julian Huppert MP alongside the named Labour and Tory candidates but didn’t name any of them in the voting intention question, is if you want to get figures that make the Liberal Democrat position look worse than it really is.
And there’s more about how the polls were conducted.
Part of ICM’s usual methodology was omitted from the polls, for reasons left unexplained. As ICM (to their credit) have made clear:
Usually, ICM would add 50% of those who refuse to answer the vote intention question or say they don’t know to the party they voted for in 2010. We did not do so on these polls.
No reason is given for doing this, however, and this adjustment usually benefits the Liberal Democrats. That indeed seems to be the case with these constituency polls.
For example, in Cambridge on the initial raw figures, Labour leads by 23 people (103 to 80). Had ICM’s usual adjustment then been made, this would have become 110 to 102, a radical shrinking of the gap.
Add then to that the impact of not using names in the voting intention question which, bearing in mind the net 58% positive rating for Julian Huppert in the poll would not have been trivial, and you see how these two methodological decisions made a big difference to the headline results. …
The combined impact of all these factors means the figures are not sensible figures for anyone to draw a conclusion from save for one: the numbers have been together in a way that causes maximum damage to the party whilst doing much to obscure what the true level of support is for the party’s MPs in the polled constituencies.
You can read Mark’s Newswire e-newsletter in full here and sign up to receive future editions here.
* Stephen was Editor (and Co-Editor) of Liberal Democrat Voice from 2007 to 2015, and writes at The Collected Stephen Tall.
Follow @LibDemVoice on Twitter
Like us on Facebook
Subscribe to our feed
Sign-up for our daily email digest
Full slate of Lib Dems in council by-elections
Getting things done - West Port #dundeewestend
Payback time
The Joy of Six 1274
Gasping Local Government.
The Chagos Islands, Mauritius and the ICJ
Honouring the Real Thing and launching Black History Month in Liverpool
Russell
Peter Martin
Peter Davies
Dave Tate
32 Comments
Not figures from which anyone should draw a sensible conclusion?
I’ll get my coat …
I find it very curious that when the Part y has just lost 91% of its MEPSand so many councillors, you are focusing on this?
Mark….do we really need any more polls after last week to tells us the sad state of this party . The doorstep spoke, it was clear and yet we have the leadership saying carry on regardless. No real plan just HOPE. Is that leadership?
These are point;ess postings and discussions. What we do have is a leader who for whatever reason is now a lame duck and all this will only blow up again early next month after the Newark by election, when I suspect we will be totally huimiliated.
Every party need motivated activists, but with the level of denial in the party and the the electoral wipeout that’s already occurred how many Lib Dem activists will be able to whip up enough enthusiasm to pound the streets next year ?
Also, how will the decimation of Lib Dem MEP’s impact on party finances ?
Let’s return to this thread in May 2015 ….
Ignoring the technicalities, one thing is clear from all the ICM polls. If Clegg goes, our vote goes up. That is clear, but LDV, the party equivalent of Pravda, won’t hear it.
I’m pleased that someone is actually looking at the detail of these polls, rather than either accepting them or dismissing them without consideration.
There may be some validity to the point about naming candidates, but on the other hand the Ashcroft polls don’t do that, do they? And I don’t remember Lib Dem commentators objecting to the Ashcroft results on that basis previously. On the contrary, Lib Dem commentators have embraced the Ashcroft results with some enthusiasm, because they have shown the Lib Dems doing better in seats they hold than in general.
As for the usual ICM adjustment of allotting 50% of don’t knows/won’t says to their previous parties, the tables indicate that would make a big difference. However, as ICM say, the tables do contain the necessary information for that adjustment to be made, and if it is made (if my sums are right) the results still show the Lib Dems losing all four seats.
That is not particularly surprising for Cambridge, Redcar or Wells, which would be lost on a uniform swing based on the national polls. But on paper Sheffield Hallam really ought to be safe even on a uniform swing basis. And of course, we don’t know whether that 50% correction bears any relation to what will happen next year. It may not.
@ Shaun Cunningham
‘No real plan just HOPE. Is that leadership?’
I think the last few days has demonstrated that there is a very clear plan – particularly now that NC has not resigned and there has been no rebellion from the MPs – many of whom will be losing their seats at the GE.
The overall plan and primary objective is to establish coalition government as a permanent feature in the UK. The leadership is banking on holding on to enough MPs to hold the balance of power after the next GE and insisting that the coalition agreement is stuck to until the end – so that the Party can be portrayed as a reliable partner when the, hoped for, negotiations begin next May.
It seems that not only NC has grown rather attached to being a government minister!
When “NOTHING TO SEE HERE” is the overriding message after a bad poll, it seems to indicate there might be something to see there.
I disagree Mark. Firstly the poll shows just 33% of people could name Julian Huppert as their MP. You seem to suggest
Secondly, all the pollsters have trouble trying to estimate Lib Dem support because so many people have turned their back on the party – You know as well as I do that people deny or forget they voted Lib Dem or say they don’t know and they are allocated to the Lib Dems as a guess by the pollsters.
Thirdly, a large amount of Lib Dems MPs “personal vote” was based on tactical voting and people voting for their second preference party because they had a positive view of the Lib Dems. That positive view has disappeared.
So we will have to wait and see how it works in 2015. Assuming it will follow the pattern of 2010 is optimistic.
Fouthly, we just have to look at the local election results in Cambridge and Wells to know the Lib Dems are in trouble.
Fifthly, if Lord Oakeshott had wished to damage the party he could have released the results before the euro elections – it is because he is trying to save the party that he did so afterwards.
I think everything is fine and we should just carry on as normal! These polls are probably nonsense, like that Euro poll thingy and the General Election.
The reality denial strategy is hilarious, and I’m starting to quite enjoy it. Caron’s had some light and that means we should talk about something else, Mark has some reservations about polling numbers 3 days after we lost all but 1 MEP, Stephen tells us 39% want Clegg gone, but that doesn’t matter because Farron/Cable are happy with him. Is this what the future holds for the next 6 years?
@Robin Wilde: I’ve detailed several ways in which the conduct of the poll set out to produce headlines that are very different from what the poll’s details really say. Don’t you think it’s wise to check details before coming to a view?
@Caractus: Julian Huppert leads 58% – 12% on good job vs bad job when people were asked (with a question that included his name). Clearly some people need reminding of his name , but when they are reminded of it – as they will be on the ballot paper in the end – he gets a massively positive rating.
“Mark Pack 28th May ’14 – 9:44pm
@Robin Wilde: I’ve detailed several ways in which the conduct of the poll set out to produce headlines that are very different from what the poll’s details really say. Don’t you think it’s wise to check details before coming to a view”
A bit like the headline “54% of Lib Dem Members want Clegg on stay on as Leader” rather than ” 54% of Lib Dem Voice Members want Clegg on stay on as Leader” . ??
Thanks Mark.
This is crude information, but priceless none the less . Having your interpretation makes the info even more valuable by an element of refinement.
Why concentrate on it? Because our MPs do need to know just how safe they are to rely on personal following and incumbency. If it makes them doubt for just a while the wisdom of doing nothing over the leadership issue and coming to a considered decision it will be of great import to the future of our Party.
“I’ve detailed several ways in which the conduct of the poll set out to produce headlines that are very different from what the poll’s details really say.”
No – you’ve suggested two ways in which the results would have been different if different questions had been asked or different adjustments used. That’s very different from saying that there’s a “right answer” which the polls were designed to disguise. Polling isn’t an exact science by any means, and the methods used vary from pollster to pollster.
The 50% correction is a good example. ICM normally does it, but it’s no more than an empirical rule of thumb. Other pollsters don’t do it. There’s no way you can say these polls got the “wrong” answer because they didn’t apply that adjustment. No one really knows whether that adjustment is appropriate or not.
@Phyllis – when there’s a poll of the public, it’s reported in the news on the lines of “35% of Brits want to eat more chocolate” or “19% of Londoners have never been to Wales” or whatever – not “35% of Brits who picked up the phone to MORI want to eat more chocolate” etc.
Saying “X% of people want Y” is the normal language everyone uses for reporting polls that are representative of the population sampled, so why would you want a special different wording for a Lib Dem Voice poll on this occasion?
So in answer to your question – no, what you suggest is very, very different.
Mark “polls that are representative of the population sampled”
But LDV specifically states that its surveys are NOT representative of the population sampled, in the small print. I feel for something so important, it should have been made a lot clearer.
“Saying “X% of people want Y” is the normal language everyone uses for reporting polls that are representative of the population sampled, so why would you want a special different wording for a Lib Dem Voice poll on this occasion? “
[my emphasis]
Perhaps because LDV polls are not claimed to be “representative of the population sampled”. This is the disclaimer that was posted after the results of the Clegg poll:
“we make no claims that the survey is fully representative of the Lib Dem membership as a whole”
https://www.libdemvoice.org/exclusive-poll-54-of-lib-dem-members-want-nick-clegg-to-stay-as-leader-40408.html
Of course that’s right. LDV polls are fine as “a bit of fun” (in the Peter Snow sense), but you know as well as I do that the raw results can’t be expected to be representative of the Lib Dem membership in the sense that would be expected of a professional polling organisation.
I have a question about the telephone sampling:
It says they get a list of all publicly available numbers then randomize the last digit to make sure they also include ex-directory numbers . So for landlines the original list they use is the landlines in that constituency. How does that work for mobile phone numbers, which as far as I know, are not assigned based on the customer’s address?
“How does that work for mobile phone numbers, which as far as I know, are not assigned based on the customer’s address?”
All the numbers used were landlines. (That used to be the case for all ICM telephone polls until a couple of years ago.)
@Chris, page 1 of the PDF suggests they also used mobile numbers although this may be an incorrect cut and paste from other polls done by the company and it lists the number of landlines called by not the number of mobiles.
If they only used landlines it is also problematic. I have been away from the UK for some time so I don’t know the situation, but in the country where I live now, polling on landlines is very unreliable (in terms of matching election results) as people who move no longer install landlines in their properties – so you aren’t talking to anyone whose economic and accommodation situation has changed much since circa 2005. Also people actually in, at home, are are less likely to have enough work and disposable income, to be unreachable.
Are there other polls anywhere which break down data based on method of contact?
For Redcar though, the real big one is that it is weighted based on census data from 2011 and the blast furnace at the steelworks wasn’t relit till 2012. So person after person is picking up the phone and saying “I work at the steelworks” and ICM sees these people as unrepresentative, counts them fractionally, but when they finally find someone who says “I used to work at the steelworks”, they multiply the value of everything that person says, as the census is telling them they should include loads of unemployed ex-steelworkers.
Obama’s birth certificate is a forgery. The evil lizards from Ickeland have rigged the poll at Redcar by reopening a steelworks. Why are those men in white coats coming for us all?
“page 1 of the PDF suggests they also used mobile numbers although this may be an incorrect cut and paste from other polls done by the company and it lists the number of landlines called by not the number of mobiles.”
I see what you mean. But I think it must be a cut and paste, as you say, because online sources say ICM generally use 15% mobiles. So the 850 landlines would correspond to a total sample of 1000.
The idea is that the socio-economic weighting will compensate for any systematic differences between people with and without landlines. Obviously if the weighting is out-of-date there’s a difficulty, but I guess the most recent census is the best that can be done.
No doubt constiuency polling is more difficult than country-wide, but surely the question is whether the inaccuracies are serious enough to change the overall picture. Perhaps the question of naming or not naming candidates could have a significant effect. But then again, that should be just as true of nationwide polling, shouldn’t it? The implication of Mark Pack’s argument is that in ordinary opinion polls, the Lib Dem vote is underestimated in 57 seats where the party would benefit from the MP being named. But in nearly 600 seats with MPs of other parties, by the same token, the Lib Dem vote would be overestimated.. The implications of that would not be reassuring for the party!
“Obama’s birth certificate is a forgery. The evil lizards from Ickeland have rigged the poll at Redcar by reopening a steelworks. Why are those men in white coats coming for us all?”
Blimey. I’m sure I wouldn’t have got away with saying that!
But I think it’s a bit unfair. Mark P and Richard S are looking at and thinking about the details of these polls, and I wish more people would.
@Chris, yes to a certain extent the census may be the best thing going, but you are effectively saying “assuming no improvements in people’s lives since 2011, these would be the results”. I don’t know about the other seats but what I posted in another thread about about Redcar seems to suggest the weighting process has seriously skewed the sample
The result of the weighting process is that the 2010 general election results in Redcar would be Labour 40%, Lib Dem 36 % – in other words, if these are the the voters then Labour won in 2010. The actual result was Lib Dem 45%, Labour 33 percent. This is in table 14 of the data.
To follow up the above, it would be then more accurate for the Guardian to say that “Oakeshott’s poll shows Redcar as a Labour hold.”
“@Chris, yes to a certain extent the census may be the best thing going, but you are effectively saying “assuming no improvements in people’s lives since 2011, these would be the results”.”
No, of course it’s not assuming that. It’s assuming no large-scale migration during the last three years that would change the composition of the constituency, in terms of the socio-economic groups that people belong to.
“I don’t know about the other seats but what I posted in another thread about about Redcar seems to suggest the weighting process has seriously skewed the sample”
Whatever it is, it’s not the “social grade” weighting process, because Labour already had a 6-point lead for 2010 in the unweighted data. Moreover the “social grade” weighting decreased the number of C2DEs, among whom Labour had a lead. It must be other aspects of weighting process that increased the 2010 Labour lead somewhat, to 9 points.
Of course, people often don’t remember accurately how they voted in the past, and in particular they often “don’t remember” voting for parties which they no longer support!
“The implication of Mark Pack’s argument is that in ordinary opinion polls, the Lib Dem vote is underestimated in 57 seats where the party would benefit from the MP being named. But in nearly 600 seats with MPs of other parties, by the same token, the Lib Dem vote would be overestimated.. The implications of that would not be reassuring for the party!”
And of course we don’t see that difference in comparisons between opinion and polls and actual election results, which suggests to me that naming/not naming the candidates doesn’t have a huge effect on the accuracy of polls.
@Chris – yes but if “unemployed” is a socio economic group then the reopening of the major employer is equivalent to the kind of large scale population change that makes the 2011 census an unreliable basis for that particular consituency. In any case, it is a very small sample but the poll puts Labour 8-5 ahead in Dormanstown ward. The area is named for Dorman Long steel and was built to house ssteelworkers – if you do a Lib Dem voice search, a Labour poster in 2011 listed it as one of Labours 4 best wards in Redcar where there was no chance for the Lib Dems (Ian Swales MP 12 cuts article). Now there was a council by-election there on euro election day which the Lib Dems won, so that also suggests the poll was innaccurate, as do the Sheffield Hallam council election results.
Just saw this thread again. Pretty funny. Looks like the polls were pretty accurate in the end….