Why the polls really got it wrong

The statisticians appointed by the British Polling Council have spoken.  The cause of the 2015 opinion poll disaster was – wait for it – statistical sampling error.  Pollsters chose the wrong mix of people.  Never mind that they had previously blamed bad statistics for their 1992 disaster, and thought they had then sorted out how to do the maths.

Meanwhile, professionals and pundits agreed that asking which party had the best leader was, consistently, a more reliable guide to who would win.  Thus, if they’d just relied on the finding that Cameron led Miliband in 2015, they’d have called it right.

Hang on, though!  If the problem was really a faulty, Labour-biased sample, then why should that sample be any less biased when asked “Who’s the best leader?”  If you have a biased sample, changing the question you ask them cannot possibly get rid of the bias!

What the “leadership” question may achieve is, instead, a truer insight into the voter’s real intentions.  So let’s try an alternative explanation, the “Shy Tory” theory.  This is also an old idea, though I’m going to give it a new twist.  A “Shy Tory” is a voter who is too ashamed to tell a pollster that he/she will selfishly vote Tory.  So he/she can say “Don’t know”.

To say that Cameron looks more like a statesman than poor old Ed is, by contrast, not actually a shameful remark to make.  So the opinion-polled voter speaks their mind.  This makes the leadership question the more reliable guide to real voting intention.

The statisticians largely dismiss this idea.  They publish a graph* showing, at first sight, that things are just all over the place.  If Tories were “shy” in 2015, why weren’t they always equally “shy”?  To a maths wizz with no political involvement, that question might seem unanswerable.

ncrm graph

However – political insight helps!  The key, I suggest, is that Tories (and others) are not always equally “shy”, ashamed, or conflicted.  It depends on what there is to be ashamed of, and that depends on the politics of the day.  The statisticians’ graph can then be read like a book.

Back in the 50s, a vote for Churchill’s Tories was patriotic, while better-off people may have felt “shy” about voting for a high-spending working-class Labour party.  So it was actually the Tory vote which pollsters then consistently over-stated.

In the swinging 60s and 70s, the graph and the political picture became more mixed.  One-Nation Tories still made a principled case for free enterprise against the dead hand of state socialism, giving their voters a story they could retell without shame.  Against this, Labour’s reforms may have created “shy” Tory voters covertly opposing sexual liberalisation, hence the polls’ understatement of the Tory vote around 1965-70.

Thatcherism, and the growth of inequality from the 80s onwards, changed things again.  With the Tory message turning increasingly mean-spirited, the numbers of “shy” Tories grew.  All seven elections since 1985 show a preponderance of “shy” Tories.  1992, when “greed is good” had become discredited yet middle-class voters still clung on to self-interest, saw a high-water mark in the “shy” Tory vote.

By 1997, the failure of Tory underspending had finally become manifest.  Blair swept to a landslide, and with only unashamed diehards voting Tory, polling error was relatively modest.  From 2001 to 2010, three elections were held with Labour in power.  Since “Throw the rascals out!” is an easy battle-cry to declaim, there were again fairly modest numbers of “shy Tories”.

The big change in 2015 was that the Tories were in power.  There was a barrage of shaming news about Tory bedroom tax, sanctioning of the unemployed, and vilification of the disabled.  An army of shy Tories who talked Left and voted Right was a predictable consequence.  Because the polls failed to identify them, they lulled Labour into false overconfidence, helped Miliband survive as leader, and helped Labour to lose.

If the statisticians drown out this message, we can only expect a repeat in 2020, with Corbyn or his successor shocked to discover a fair-weather Labour “vote” which runs away on election day.  This will gift the Tories five more years hegemony, and leave Liberal Democrats in the wilderness.  It matters, therefore, that this message gets through.

*Note: The graph above is from slide 10 of reference 1 in this document list.

* David Allen is a member of the Rushcliffe Local Party and has been a member of the Lib Dems or its (SDP) predecessor since 1981

Read more by or more about .
This entry was posted in Op-eds.


  • “Meanwhile, professionals and pundits agreed that asking which party had the best leader was, consistently, a more reliable guide to who would win.” ….So what does that say about Cleggie, then ? The plot that failed (see Liberator) because of lack of bottle ought to have a good look at themselves.

    Another bit of the article says : “In the swinging 60s and 70s, the graph and the political picture became more mixed. One-Nation Tories still made a principled case for free enterprise against the dead hand of state socialism, giving their voters a story they could retell without shame”.

    Oh no they didn’t. As someone who joined the party over twenty years before the poster of this article…. (had a thoroughly good time as a ‘Red Guard’)…. twas not so. As Max Boyce used to say, ‘I know ‘cos I was there’. Apart from steel nationalisation there was little of the ‘dead hand of socialism’ about Uncle Harold…. Alec Home/Ted Heath were about as exciting as a lump of lard and under Thorpe we had a leaflet of Wilson/Heath with the title ‘Which Twin is the Tory’. Then – as it ought to be now – Tory was a widely accepted term of derision.

    The future ? Too soon to tell. If the Tories choose Osborne and Jeremy C. falls on his sword to be replaced by the dream team of Dan Jarvis/Stella Creasy, who knows ??

  • Simon McGrath 1st Feb '16 - 10:10pm

    “There was a barrage of shaming news about Tory bedroom tax, sanctioning of the unemployed, and vilification of the disabled.”
    well except that polls consistently shows that welfare cuts are rather popular ( alas)

  • Richard Underhill 1st Feb '16 - 10:50pm

    There is another question: Who has the best policy on the economy? which may feed into answers of the other questions.

  • Polling accuracy, like canvassing accuracy, needs to same questions to be asked of the same rectors more than once. Indeed they probably need to be put five or six times to get an accurate picture of how people will vote. It couldn’t be, could it that voters are simply sticking two fingers up at pollsters and canvassers?

  • David Allen 2nd Feb '16 - 12:21am

    Simon McGrath, you’re missing my point. Welfare cuts may indeed be popular, and may indeed be part of why the Tories won. However, some people will privately vote in favour of such policies, but are ashamed to say so openly to a pollster. Hence the gap between the real vote and the opinion poll.

    I’m just not making a politically partisan point in this article, capisce? I’m quite happy to return to knockabout next time around, but, here I’m just seeking to understand what is going on with polling.

  • David Allen 2nd Feb '16 - 12:30am

    David Raw, I’m not clear why you can so confidently declare that in the 60s and 70s, the Tories did not make a case for free enterprise versus the dead hand of state socialism. I think you are saying that, because you personally thought that Harold Wilson was more dynamic than Home and Heath, therefore the Tories could not have sought to make that case. Well, sorry, but that is the case they sought to make, with Heath’s “Action Not Words” slogan for example. All you’re telling me is that it didn’t convince you and it didn’t win your vote. Millions of others could say the same.

    Let’s put it another way. I take it you would accept that there are many things in Conservative philosophy and governing practice which ought to make decent people ashamed to vote for them. Very well. Do you think this got worse under Thatcher and her successors, or did it just stay the same. I’m arguing that it got worse, and that this is why the polls began to show increasing numbers of “shy Tories”. Do you agree?

  • David Allen 2nd Feb '16 - 12:38am

    Peter Martin,

    Broadly, your analysis looks at similar questions to those considered by the Inquiry statisticians. There is some sense in what you, and they, are saying. But basically, what it adds up to saying is that, because things like voter willingness to vote change in unpredictable ways between one election and the next, the pollsters are always likely to get it a bit wrong, in totally unpredictable ways.

    However, the actual reality is that the pollsters often get it nearly right, but sometimes – 1992 and 2015 – get it spectacularly wrong. Consequently, it is worth considering whether the results might not be totally unpredictable, and whether in fact there might be a rational explanation. I think there is such an explanation.

  • David Allen 2nd Feb '16 - 12:44am

    Flo Clucas, yes it’s quite likely that a small proportion of voters might deliberately lie to pollsters just for fun. But why should they have done that in droves in 1992 and 2015, but far less at any other recent election?

    If we are to understand what went wrong, we need a rationale that explains what was special about those two elections (and why other trends, such as the anomalous opinion polling bias toward Labour instead of the Tories in 1945-1955, were also observed). I have sought to put forward such a rationale. Might it possibly make sense?

  • They got it wrong because they played it as a normal election. The reality is the Tory vote went up by under 1 per cent, but because the Lib Dem vote collapsed this meant they took more seats than expected. The Labour vote was up on 2010 despite losing Scotland. No one was expecting us to be reduced to 8 seats, most were predicting anything under 30 as highly unlikely. Same for Labour in Scotland. Vote splitting meant that where the Tories were in a strong second place they took the seat.
    In short the polls were underplaying anything that didn’t fit the patterns of previous elections. If you take out our collapse and the rise of the SNP the polling wouldn’t be that far off. There is simply no other way to explain the Conservative majority based on the negligible rise in their vote.

  • “No one was expecting us to be reduced to 8 seats, most were predicting anything under 30 as highly unlikely.”

    But that’s not really true. In fact, if you took the polls seriously, you had to conclude that the Liberal Democrats were bound to come in under 25, with an even dozen as a very likely possibility. Anything less might have seemed unlikely, but not impossible.

    Only in places where a clear-eyed and hard-headed assessment of the situation was taken to be tantamount to treason were numbers above 30 MPs treated as if they were reasonable possibilities.

  • George Kendall 2nd Feb '16 - 6:30am

    We mustn’t imply that soft Tories are morally inferior to us. I’m sure they don’t think of themselves that way. And they won’t vote for us if we speak that way.

    So rather than talking of ashamed Tories, we should talk about shy Tories. And shy, perhaps, because some left of centre politicos imply voting for anyone but left of centre is morally reprehensible.

    Most people don’t respond well to what they regard as moral blackmail. So attempts to morally pressure voters into not voting Tory (and there were a lot of such attempts) may even have increased the Tory vote.

  • David Evans 2nd Feb '16 - 8:25am

    An interesting article, but the final paragraph only provides an answer to the wrong question. It won’t be the nasty statisticians that lead to Labour collapsing and the Lib Dems being left in the wilderness, but because neither Labour nor the Lib Dems have dared to face up to its failures over the coalition years.

    Just urging the true believers to carry on believing and working hard while focussing on nice things in our comfort zone like transgender history and all women shortlists will do nothing but cement us in basement for voter. We need to reconnect with real voters like we did in the past and not just look to placate vociferous party minorities. That requires a real willingness to change and address the trust issue.

    Until we start to get ordinary voter’s trust back, we will remain on a downward spiral. Is anyone at the top of the party saying this or are they all too proud to actually admit to getting it wrong in the past?

  • Surely the point for us Lib Dems, is, as David-1 has said, the pollsters got it right, in general. Where they got it wrong was in the precise balance between Labour and Tory, especially so in some marginal seats in England. Lib Dems should not accuse the pollsters on this, we should look back to the debate we have been having throughout the period of coalition (on LDV as much as anywhere), that there “would be a recovery”, and if there wasn’t, we would be saved by the amazing popularity of our MPs locally. I only take limited pleasure in saying that it was always pretty obvious, when our polling failed to increase, that “popular” MPs would go the same way as “popular” councillors and MEPs. The other thing that made it a worse result for us is that to some extent, the national political commentariat were swayed by the spin our spokespeople were able to put on this – we had always had a higher “incumbency effect” etc. And, I think, at the top, we believed our own propaganda, and that is ALWAYS dangerous.

  • Glenn is also right, but the polling figures did point to our huge drubbing, and the SNP’s huge gains. So perhaps, as others have suggested here, it was differential turnout between Tory and Labour which did it? That is not too far from the BPC’s conclusion that they didn’t interview enough Tories, or even David Allen’s theory that there were more shy Tories than the pollsters’ weighting adjustments allowed for? It is always known, for instance, that Labour leaners (and ours too) turn out much less reliably that Tories. Anyone who has done practical getting out the vote factors that in.

  • David Allen asks :
    “But why should they have done that [ lied ], in droves in 1992 and 2015, but far less at any other recent election?”
    Good question.
    If you are (say), a Labour person, you probably mix with family and friends who are also Labour. So there is an unspoken ‘herd expectation’, that you will continue on the Labour track (peer pressure). However, if that person starts to take on a different message from Tories, or Ukip, or Greens etc, only a strong willed and confident person will acknowledge their shift in thinking amongst their family and friends. If you know that your shift in politics will hurt or even anger your relationship with friends and family,… you will lie,… right up to the sanctuary of ballot box. As the old adage goes.. never talk religion and politics with friends!
    As to why some times and not others, this ‘closet voting’, is more likely around the time of some new [and intriguing], kid on the block,.. Thatcher,.. Blair,.. Farage,.. Trump are examples of politicians likely to produce ‘closet voters’, who’s ‘secret cohort vote’ will surprise everyone, because they never let on, to friends, family,.. and pollsters. In fact they may still be lying to friends, and family…”Tories won,.. shock horror,.. nothing to do with me Guv,… honest”
    If you think intentionally ‘closet’ lying [as a cohort group], to pollsters is farfetched, let me throw you another curved ball.
    SNP voters are still smarting from losing their Scottish Independence referendum. Worse than that, it could be a generation (20 years?), before they get another go at independence. However they [SNP], have also been told that if the EU referendum goes ‘Leave’, then a new Scottish Independence referendum will be accelerated forward. Might it not be very tempting for SNP voters, to tell pollsters they intend to vote ‘remain’ in the EU referendum, but actually vote ‘leave’ for tactical SNP independence reasons?
    So in short,..The ‘herd expectation’ is that SNP voters will vote ‘remain’ in the EU referendum, but do not be the least surprised if there is a massive ‘leave’ EU vote in Scotland, because a *sacrificial leave* vote will pull SNP voters, closer to their holy grail,… Scottish independence ?

  • David Wright 2nd Feb '16 - 11:30am

    Yes, Glenn got it right, but that’s not the full story. There’s another key point, and I don’t know why the pollsters don’t acknowledge it: the way the Tories operated against Lib Dem held seats.

    We know that in marginal seats, a quite small number of ‘soft’ voters who might go either way will decide the outcome. What the Tories did was identify (most of) those voters in LD seats and target them ruthlessly. It only took a few hundred voters switching in a score or so of seats to give the Tories their overall majority.

    In polling terms, a change of about 0.1% of voters made a 20 seat difference to the result. A change so far below the statistical margin of error in normal survey. To correct this, the polsters would have had to know who the Tories were targeting and where, then do extra polls in all those seats, asking at least 1000 voters in each constituency. Or else grow their random sample from 1 thousand to 50 or 100 thousand voters. Massively expensive, and it would take too long for the news cycle anyway. Never going to happen.

  • David Allen 2nd Feb '16 - 11:43am


    It’s great to see someone really reading the post properly and engaging with what it argues!

    On this question of “lying” to pollsters, I would point out that for the most part, actual “lying” is not required. We are very used to the category “Probably” in canvassing, aren’t we? It’s what lots of people say on the doorstep. But the pollsters won’t let you say “Probably”. You can either plump for “I’m voting Labservadem!” or you can say “Don’t know.”

    To get a faulty result in opinion polling, therefore, all that is necessary is for a cohort of Tory-Probable voters, who are feeling rather conflicted or ashamed about a Tory vote, to plump for the “Don’t know” option. Indeed, it may also be that a cohort of Labour-Probable voters, who have always been Labour but don’t really think Ed M can cope, will plump for “Labour” in the opinion poll, but then not feel motivated enough to get along to the real poll and vote. None of these people are actually lying. Faced by the pollster with a choice between two half-truths, they are choosing the one that feels better at the time.

    Interestingly, the exit poll, though much more accurate than the pre-poll opinion poll, still overstated the Tories by about twenty seats. I think it is the exit poll which picked up on the rather smaller number of Tories who are so “shy” that they will actually tell a direct lie about the way they have just voted.

  • David Allen 2nd Feb '16 - 12:07pm

    Glenn, David Wright: I’m afraid that the polls got the Lib Dem vote largely right, and their Lib Dem figure hardly moved one way or the other throughout the campaign. So, the Lib Dems who made pre-election predictions of 30+ seats were just kidding themselves. Never believe your own propaganda!

    On this question of changing opinions – I think the opinion polls are a bit like a thermometer in which the bulb and stem have slipped out of their housing, and moved up or down the measurement scale by an unknown amount. So, if the reading is “20 degrees”, you don’t know whether to believe it. Because of the unknown slippage (or in polling terms, the unknown change in weighting factors caused by changes in “shyness”), it might in fact be 16 degrees, or 23 degrees. However – Importantly, if that thermometer starts to rise, you DO know that it is getting hotter! So – What the polls do tell us, most reliably, is whether a campaign is really shifting votes – or not.

    The Tories who threw the book at the Lib Dems in the south-west would love to believe that it was their campaign wot wun it. So they trumpet that success. Don’t you believe it. The polls hardly moved throughout the campaign. The chances are that well before it started, the Lib Dem vote had already fallen to the point that the Tories were on course to win. By and large, our 23% in 2010 had fallen to 8-10% by 2011, and it just stayed there until 2015.

    Yes, I’m a scientist. Science matters. If you don’t start with a proper understanding of polling science, then you don’t know what planet you are on, and you can’t make good plans as to how you will seek to advance its politics and its government!

  • David Allen 2nd Feb '16 - 12:28pm

    David Evans: “The (article) only provides an answer to the wrong question. It won’t be the nasty statisticians that lead to Labour collapsing and the Lib Dems being left in the wilderness, but because neither Labour nor the Lib Dems have dared to face up to its failures over the coalition years.”

    Well – Understanding why everyone thought Labour had a good chance, when in fact they didn’t have a hope in hell, is hardly a “wrong” or insignificant question. The statisticians have basically told us that, once the pollsters have corrected a few technical mistakes, everything will be fine, and they won’t get it horribly wrong next time. That’s exactly what they said in 1992. They were wrong then and they are wrong now. Seriously wrong.

    What the pollsters told us in 2010-2015 provided Labour with a spurious comfort blanket. It encouraged their lazy faith in a “35%” strategy – that they could win with a smallish core vote, and they didn’t need to take any risks (like, come off the fence and offer some real policies!) to expand that vote. The polls encouraged Labour wishful thinking, and they didn’t need much encouragement.

    The Lib Dem case was even worse, of course. The polls gave the Lib Dems no false encouragement at all, and yet the Lib Dems stuck themselves in a comfort zone. Like the Hitch Hiker, the Lib Dems wound that black towel tightly over their eyes so that they could see no danger, and refused to panic!

  • David Allen
    That’s not what I remember. I can remember numerous interpretations of the polls on TV and in the press suggesting around 30 seats and not one suggesting 8. But I agree with the general thrust of what you are saying and suspect we are actually saying similar things. The point I was making is that whilst the polls were actually closer to being right than some suggest the interpretation was out because they were being skewed by caution and trying to make the stats fit previous election results . One of the big mistakes made by the Lib Dem leadership was to put too much faith in Ashcroft’s poll of polls

  • David Allen 2nd Feb '16 - 12:47pm

    George Kendall, “Most people don’t respond well to what they regard as moral blackmail. So attempts to morally pressure voters into not voting Tory … may even have increased the Tory vote.”

    Yes, you have a point, though one can also argue the opposite. Moral pressure has played an important role in reducing things like drink-driving, hooliganism, and overt racism, sexism and homophobia. Winning a moral argument can also put the losing side politically on the defensive, make a vote for that side “not cool”, and win votes for the side which can validly claim the high moral ground.

    People who do sh*tty things don’t like to be called sh*ts. If you avoid calling them sh*ts, you may get their votes. Trouble is, you also tend to end up endorsing sh*tty policies. You then face a moral choice.

  • @ David Allen ” I’m not clear why you can so confidently declare that in the 60s and 70s, the Tories did not make a case for free enterprise versus the dead hand of state socialism”.

    I didn’t say that. I said the majority of the electorate (not just me) didn’t see much of ‘the dead hand of socialism’ about Uncle Harold.

    He was seen as the moderniser (rightly or wrongly) in touch with ordinary folk. For an intellectual Oxford Don, he cleverly came across as a down to earth competent northern lad from Huddersfield – carrying a picture in his wallet of the 1926 Town team that won the title (I know that. He showed it to me in the Scilly Isles when I queued up behind him in the Co-op and he was happy to chat ) – unlike when I once bumped into Heath who came across as a cold fish.

    The Tories came across as a bit boring with not much to say (apart from Ted sailing his boat with young women on board for some reason or other at the instigation of Central Office).

    Agreed things got sharper and moved to the right under Thatcher (influenced by the dotty Keith Joseph) – but in 1970 Uncle Harold lost the election rather than Heath won it (we went down to six MP’s). Macmillan was the last one nation Tory PM. Home/Heath never really resonated with the wider electorate – as the two 1974 elections showed when Harold bounced back.

    2. There’s only one real poll – the one in the ballot box. We got hammered this time because we were seen as no more than an annex to the Tory Party (for the first time since 1950).

    3. I find early nights help these days !!!

  • David Allen 2nd Feb '16 - 3:46pm


    Yes, Wilson portrayed himself as a moderniser, as did Blair after him, in both cases with a fair amount of personal success. However, I think that both were acting to counter a powerful, long-running and ultimately successful campaign by the Tories to tar Labour with the brushes of statism, semi-communism, bureaucracy and inefficiency.

    From my perspective, the old-style Tory campaign for free enterprise against socialism can be described as principled. Ordinary voters could therefore happily and openly declare, to all and sundry including pollsters, that they were voting Tory to get competent and (supposedly) non-ideological government. That’s why they weren’t such “shy Tories” in those days.

  • David Allen 2nd Feb '16 - 4:02pm

    Indigo and others, on the question of opinion polling and the EU referendum, see:


    Online polls put “Remain” and “Leave” virtually neck-and-neck. Phone polls give “Remain” a massive lead of almost 20%. Nobody knows which, if either, is right. I can’t easily put this one down to the “shyness” factor. I certainly don’t think that the statisticians have it solved, either.

    It matters! Hedge funders are conspicuous amongst those who have promoted the OUT campaign, perhaps because they stand to gain financially from turmoil in the markets. If they can pay for a more reliable predictions of future events than are available to the rest of us from inadequate opinion polling, then they can potentially make a killing. At the expense of good governance, of course.

  • David Allen 2nd Feb '16 - 5:30pm


    Hmm. Let’s try, for example, a policy of cutting benefits to the disabled. The excuses made by those who favour such policies include: that some of the claimants are dissembling or scrounging; that we have to prioritise cutting the deficit; that money needs to be diverted toward promoting employment, which if successful will help the disabled find work; that there are many other deserving causes competing for government funds. Undoubtedly there are elements of truth in all these excuses. How do we proceed?

    We can begin, as you suggest, by avoiding the personal. We can simply speak up for the downtrodden (as, in parallel circumstances, Tim has done from Calais). What generally happens is that the Right then make it personal. They get the boot in first, and talk about sanctimonious soft-headed liberals who don’t understand ordinary people, have no respect for hard work, wouldn’t recognise a scrounger when they saw one, couldn’t run a chip shop, and can’t be trusted with our money. Then it’s personal, and it’s moralistic, and we are on the losing end of the argument, because we didn’t have the courage to say what we meant. We can respond belatedly, with disdain which appears as weakness, or with aggression which belies our initial soft-speaking, or with rational explanation. Then “When you’re explaining, you’re losing.”

    That’s how Miliband played it, and that’s why he was adjudged to be weak. That’s how Gore lost his debates with Bush, despite his much greater intellectual capability, in which he placed far too much faith.

    The “nice guy” schtick can work in local politics, where many Lib Dems adopt it, though even at that level the deliberately nasty guy can take aim and beat you (and NB, that comes from personal experience!) In national affairs, I don’t think just being “nice”, and seeking “a way around moral choice” has worked, or will work, for our party.

  • David Allen
    The shyness factor plays very little part in British elections, pundits were just mis- reading of the polls because they were not taking into account how this stacked up seat by seat. The Conservative Party picked up under 1 per cent of new votes since 2010 and when you take into account a low turnout this means they got about 24% of the electorate. The real shyness factor is the 40% , those I suspect, who don’t feel anyone is worth voting for.
    The dirtiest word in politics is populist because none of the main parties really seem to believe they have to represent the electorate.

  • David Allen 2nd Feb '16 - 8:00pm

    Glenn, follow the hyperlink instructions in the footnote to my article, and you’ll see that all the pollsters do accept that they got it wrong, with an error of 6.6% in the (Tory minus Labour) vote. Yes, it can also make a difference how that stacks up seat by seat, but that’s a separate, subsequent calculation. It doesn’t alter the fact that the polls got the voting figures badly wrong. It’s nothing to do with how pundits read the figures. It’s that the figures were wrong in the first place. An explanation is therefore necessary.

    The statisticians have done what they are qualified to do, which is to crawl all over the statistical design, and look for flaws. So they have found some. For example, pollsters just use one “over 65” category. That’s a bit duff, because a lot of “youthful” 65 and 66-year olds answer the phone to pollsters, whereas 75-year-olds by and large don’t bother talking to pollsters, but do vote. So hey, they have found a potential source of bias! What (on the whole) they can’t explain is, why have these statistical bias factors suddenly become huge in 2015, when they hardly mattered at all in 2010, 2005 and 2001?

    What statisticians can’t cope with are “soft” variables, such as how shameful it might be to admit to a Tory vote in a particular year, with particular issues swaying the voting decisions. Soft variables can’t readily be quantified, and to assess their merits, there is no alternative to subjective judgment. Statisticians don’t like that, which is to some extent understandable. However, ignoring soft variables just because they’re difficult to analyse is a big mistake!

  • David Alan,
    The point I was making is that on the voting numbers if you take out the Lib Dem collapse it was another hung parliament. My second point is that more people do not vote than vote for any single party.

    Further more rather than Shyness/shame is it not possible that some people simply changed their mind because the economy looked healthier? I don’t dispute the figures, although without losing Scotland Labour would have had around 270 to 280 seats and did have an increased vote on 2010.

    Personally, I have a problem with attributing untestable mass psychological theories as a key factor in anything. You don’t know if people were shamed or changed their mind or anything else without sitting in their brains. What I do know is that the Conservative vote didn’t rise by enough to explain the change in seats. where as the collapse of the Lib Dems does seem to explain the shift. To be honest I think the “shamed ” Tory idea is a crock of faux victimhood perpetuated by right wingers. Where does this alleged pressure come from? The press?

  • David Allen 3rd Feb '16 - 12:00am


    I agree with most of your latest. I certainly don’t advocate acting “nasty”. I do advocate making clear moral choices and making moral reasoning clear.

  • David Allen 3rd Feb '16 - 12:05am


    “Is it not possible that some people simply changed their mind because the economy looked healthier?”

    Yes, of course that’s possible, indeed it’s probable that this happened to some extent. But, but, but – Why do you suppose that when Joe Bloggs decided that the economy was picking up, he went and said “Don’t know” to the opinion pollster, and then went and voted Tory straight away afterwards?

    That’s what has to be explained – The 6.6% gap between the opinion poll figures (specifically, those obtained as close as a day before the election) and the actual vote percentages.

  • David Allen.

    Maybe they interviewed people who didn’t know until they voted, maybe some of them ended up being from the 40% of the population that didn’t vote at all, maybe the sample was too small to accurately predict the actions of millions of people. To me 6% is within the margin of error.
    What I do know is that in most of the seats we lost to the Tories our lost support was way larger than votes picked up by the Conservatives so they actually won by default in those seats and in more than a few cases the Conservative vote actually went down and they still won the seat.

  • David Allen 3rd Feb '16 - 10:49am

    Peter Martin, “I’d be surprised if you can legitimately draw this conclusion.”

    What you mean is that you think (though neither of us has actually done the maths) that one cannot exclude the possibility that the twenty-seat error was purely due to random polling error, rather than a systematic cause such as a few percent of people lying. Statisticians, as I expect you know, use various arbitrary thresholds to determine whether or not a hypothesis should be adjudged “legitimate”. Thus for example, if the maths showed an 8% probability that a result was due to pure random variability, and a 92% probability that it had a systematic cause, then most statisticians would say that it was not “legitimate” to claim that one can be sure there was a systematic cause.

    A professional gambler, however, would race down to the bookie and put money on the “horse” which had been shown to have a 92% chance of winning!

    In politics, we’re in the same position as the professional gambler. We want to do things which are likely to enable us to win. We don’t require mathematical proof that we will win if we do X. If we wait for that, we’ll do nothing and lose.

    That exit pool was quite a long way out considering its methodology. You could see that the (good) statisticians who devised it were surprised by how far out it was. I think that on the balance of probability, the error was likely to have been due to the “shy Tory” factor, and that to ignore that high probability would be, in a sense, negligent.

  • David Allen 3rd Feb '16 - 10:57am

    Glenn, “To me 6% is within the margin of error.”

    I’m afraid that is a comment that verges on denialism. All the professional pollsters acknowledge that it was not within the margin of error, that they should have done better, and that an explanation for the bad performance needs to be sought. If you’re just going to ignore professional opinion, and say you know better, then you’re doing much the same as the climate change denialists do with the evidence of the professional climate scientists.

    What that explanation is is far more debatable. My explanation could be wrong, or only partially right. The statisticians may have correctly identified some of the factors involved. There is a debate to be had – but not a debate with denialism.

  • David,
    You’re just determined to see a “shy Tory” factor. I grew up in a very tory suburb, I never met a shy Tory. And again where does this shyness come from. They’re certainly not shy in the Press or on the Internet or in my local. There’s far more shaming and name calling of “liberals”, “liberal elites”. “do gooders”, “lefties” etc.

    The thing about exit polls is that the numbers involved are much greater and they are on the ground, plus they don’t involve many people who might not be voting at all. You yourself point out that the polls seem to be weighted in favour of younger people. Well, younger people are less likely to vote. Maybe, that’s what they’re shy about. I don’t know, because I can’t read their minds and nor can you or Yougov or anyone else.

  • David Allen.

    I think professional pollists with work with a small, in comparison to the millions who vote actually vote or don’t, and it throws up anomalies. They tend to get it right because the results are usually fairly predictable anyway. If you take your analogy of gambling you can still lose even on what looks like a safe bet. It doesn’t mean the general system is wrong. It just means that the result in that instance was less predictable than it seems. In 2015 election the main factors were the Lib Dem collapse in tory facing seats , which had not happened on that scale in decades. and for Labour wipe-out in Scotland.
    In what way is this denialism and is certainly nothing like climate denialism, because in climate science you are not trying to measure a mood or attributing motives to the climate. And of course climate models are often off even though the general data is accurate.

  • @ Glenn,
    I don’t think you’ve really grasped the concept of what David means by a shy voter. Maybe ‘shy’ is the wrong word. Maybe ‘closet’, or some other word might be more useful. Also this ‘shyness’ [or whatever we call it ], doesn’t just apply to Tories.
    You say :
    “I grew up in a very Tory suburb, I never met a shy Tory.”
    What you fail to ask yourself, is that within that Tory suburb, how many Tories told their friends and family [ to avoid upset ], that they intended to vote Tory, but in fact voted Ukip? Ergo, they were ‘shy Ukippers’?
    This is not about shyness, it is about the psychology of wanting to fit in with the ‘expected norm’, of your particular environment, whilst secretly seeing the world differently. And lying,..simply to appear to fit in?
    To expand this point :
    Imagine a typical Tory pub where everyone is having a drink as friends. Do you think Giles is going to suddenly open up and admit that he is going to vote for the Green Party? (in a Tory pub, amongst Tory friends !!!!!)
    Giles is going to keep shtummm, because Giles is [understandably!], a ‘shy Green’, amongst his Tory mates.

  • David Allen 3rd Feb '16 - 2:04pm

    “I never met a shy Tory. … They’re certainly not shy in the Press or on the Internet or in my local.”

    The vast majority of Tories are “out of the closet”, and as you say, vocal and often strident about being Tories. But that’s not the point. It only takes a small minority of Tory voters, who are wavering and conflicted about their vote, to produce significant polling errors. In this election, a 6% error made a huge difference to the final prediction and is acknowledged by the pollsters as a deadful result. So don’t think about the loudmouth Tories. Think about the quiet, apologetic middle-class guy on the doorstep, who stares at you with a hangdog expression, and says he quite likes your leaflets and might possibly vote Lib Dem – You’ve met a shy Tory!

    “You yourself point out that the polls seem to be weighted in favour of younger people.”

    No I didn’t say that. I said that one of the rather small flaws which the statisticians had found was a bias toward polling 65-66 year-olds at the expense of 70+ year olds. It’s a real flaw, but the statisticians did not produce any evidence that it had a signifgicant effect on their overall figures. So, it’s nothing at all to do with 20-year-olds who don’t vote. Sorry, but if you’re going to discuss a rather technical subject like this one, you do have to read what people say.

  • Indigo,
    It was a tory suburb it was normal to vote Tory and to talk about it.

    To be honest I think the whole concept of “shy” voters is the Deus ex Machina of pollsters. It’s entirely unprovable and arrives just in the nick of time to explain why their predictions were out. Well, the God of political shyness intervened, next time we’ll get it right. I get the concept. I just think it’s a crock of baloney.

    Not talking about voting intentions to fiends in a pub is not the same as not taking to pollsters in private. Personally I suspect, they interviewed far far less than 1% of the population and because it’s a relatively small number of the millions involved in voting it throws up anomalies until you get the exit polls which involves far greater numbers and excludes people who might not even be voting. To me polls look accurate when the result is already predictable and in this case they were basing it as much on the 2010 election as anything else. Without out the collapse of our vote there was another hung parliament. The collapse in our vote was a much bigger factor than mythical Tories and is what we should be addressing rather than anything else. And it was not a big win anyway.

  • And David Allen
    I’ve never handed out a leaflet in my life except for gigs and even if I had how do you know “the quite apologetic middle class man” doesn’t mean exactly what he says or has no intention of voting at all. Statistically, there’s nearly twice as many non voters than those who vote for a particular party. I don’t know because I can’t read minds, nor can you or anyone else. You are inventing a person to sell a scenario you have no evidence for.

  • Glenn,
    “It was a tory suburb it was normal to vote Tory and to talk about it.”
    Well they would wouldn’t they? That’s the whole point of being a shy/closet voter,.. i.e. to talk ‘like a… [Tory], …[Labour], …[LibDem]…..’, but secretly be of another political persuasion?
    The principle is not dissimilar to being closet gay, or bisexual. It’s well accepted that there are very likely more (say), gay footballers than admit to it.
    It’s not a far stretch then,..to suppose that those Tories that you know, are [or might be!], hiding their real politics, in the same way [ and for much the same reason], that they might be hiding their real sexuality?

    General question to anyone listening :
    Just out of interest, does anyone know of a legal reason why a polling company could not just convert a Transit van with half a dozen secret booths and a ‘pretend ballot box’, and park it on a variety of town precincts up and down the country. Then just hand out dummy general election style, ballot papers, to eligible adults, and see how a genuinely secret, but dummy ballot, pans out?
    Is the pollster actually asking questions [out in the open], the problem?

  • Indigo.
    I get the idea, but I don’t buy it for the reasons stated. Mainly because it implies that admitting to being a Tory is a dark secret in a hostile environment and in this scenario we are taking quite specifically about shy Tories. Secret Ukipper , I can just about believe or something along those lines. But secret Tory in a Britain with a Conservative Government, a very vocal Conservative Press etc. I think it’s at best far fetched.

  • David Allen 3rd Feb '16 - 5:18pm


    “I have a problem with attributing untestable mass psychological theories as a key factor in anything. You don’t know if people were shamed or changed their mind or anything else without sitting in their brains.”

    Now that’s a perfectly fair point – Up to a point. It’s always more satisfactory if you can find a testable explanation for a phenomenon, which is why people usually try to do that first. They latch on eagerly to anything testable which they can find. So, with the age bias problem I have mentioned above, the statisticians and pollsters would love to be able to believe that their 65-year-olds, which they now realise they had over-sampled. would have been far more Labour-inclined than their 70+-year-olds which they had under-sampled. However, if you click my links and read the references, you’ll see that they can’t offer any actual evidence about that. I think that is basically because it just isn’t a big enough flaw to be remotely likely to explain the size of the polling error.

    So what do you do, when you can’t find a testable explanation for a phenomenon, and the only plausible explanation is something that you can’t test? Do you just give up, and say you’ll cope without trying to work out what is going on? I don’t think you should do that. In politics, you don’t have mathematical proof that (for example) Corbyn will lead Labour to defeat. You use intuition – you try to sit inside voters’ brains in fact – to decide what you think on such questions. You should do the same with the opinion polling error question.

  • David Allen 3rd Feb '16 - 5:43pm

    Indigo, “Is the pollster actually asking questions [out in the open], the problem?”

    Possibly, but I suspect not. My first guess was that a phone poll, where you have to talk to a real person, would provoke “shyness”, whereas an online poll, where you click a box and feed an anonymous data point into a computer, would not. However, in GE 2015 at any rate, these two types of poll were equally in error.

    To elaborate on “shyness”: Actually, any canvasser knows that many voters (say,
    something like 30%) genuinely haven’t made their minds up when questioned, and are wrestling with the choice. These are actually far higher figures than the GE 2015 6% error in (Lab minus Con). Let’s simplify the voters’ dilemma into “I think the Tories are best for my self-interest but they are nasty and uncaringto the weak, so which way do I jump?” Now, let’s suppose that when opinion-polled, this 30% splits 18 to 12 in favour of “Dammit, I’m going to do the decent thing, and vote Lab/Lib!” Then let’s suppose that in the real polling booth when the chips are down, the same 30% split 18 to 12 in favour of “Jeez, I can’t afford not to vote Tory!” Well then, there’s your “shy Tory” effect. As you point out, “shy Tory” is likely to be a considerable simplification of what is happening, but as a rough guide to what is (I think) happening, it will do for now.

    You’ll note that in this version of the “model”, it really, really isn’t about lying, as such. Possibly it’s just a little about self-deception. It isn’t just “Tories” either. It’s waverers, quite a few of whom won’t end up voting Tory.

    You’ll also note that in this version, the critical question is whether you are voting for real or just doing a dress rehearsal. People often say in many situations, rightly I think, that they won’t really know how they’ll act until they have to do it for real. If that’s so, then your idea of doing a staged mock-up won’t get any closer to reality – Sorry!

Post a Comment

Lib Dem Voice welcomes comments from everyone but we ask you to be polite, to be on topic and to be who you say you are. You can read our comments policy in full here. Please respect it and all readers of the site.

If you are a member of the party, you can have the Lib Dem Logo appear next to your comments to show this. You must be registered for our forum and can then login on this public site with the same username and password.

To have your photo next to your comment please signup your email address with Gravatar.

Your email is never published. Required fields are marked *

Please complete the name of this site, Liberal Democrat ...?


Recent Comments

  • User AvatarDavid 21st Aug - 5:57pm
    Good article, but disagree the Liberal Democrats don't have radical policies - we do need to push them more - ending the Benefits Freeze, will...
  • User AvatarDavid Raw 21st Aug - 5:44pm
    @ TCO "Yet in 1980 Jo Grimond wrote “Liberals must stress at all times the virtues of the market, not only for efficiency but to...
  • User AvatarMichael BG 21st Aug - 5:24pm
    TCO, On the 19th August you suggested five policies which I assume think should be the core of our message once Brexit has been dealt...
  • User AvatarDavid Raw 21st Aug - 5:23pm
    @ Geoffrey Dron Yes, Geoffrey, I've read the Due North Report..... not surprising given I was brought up in West Yorkshire. As a former Convener...
  • User AvatarInnocent Bystander 21st Aug - 5:21pm
    Joe, Elliot is right on the problems but offers the usual trite and feeble solutions in response. "Invest in infrastructure and whatever is the latest...
  • User AvatarJohn Marriott 21st Aug - 4:39pm
    @Tom Harney Bonn became the provisional capital of West Germany as it was hoped that reunification would take place within a few years, with Berlin...
Sat 24th Aug 2019
Thu 29th Aug 2019
Mon 9th Sep 2019