How did the media do at reporting opinion polls in January?

As I blogged last month, The Voice is going to start rating the quality of the media’s coverage of opinion polls, which is often far from perfect:

There is progress, helped no doubt by the criticism from Anthony Wells and Mike Smithson, both of whom are respected by many of the relevant journalists.

However, there is still much more that could be done to raise the overall quality of such reporting, so here at The Voice we’re going to start scoring each poll commissioned by a traditional media outlet and the way in which its initial report is worded.

Once the scores have been running for a little while, we can have the fun of a league table showing the best/worst journalist/media outlet at reporting opinion polls. Fun – but also a way to encourage those who want to meet high standards to raise their game.

I’m intending to update the scores each month, broken down by newspaper title. We’ll (only) be scoring newspapers on their reporting of polls they have commissioned themselves. That’s partly for practical reasons and also because accurately reporting your own news is a basic the media should get right.

So here are the first round of scores (for the calendar month of January) out of a maximum of 30 points:

1= Sunday Mirror: 25
1= The Times: 25
3 Independent: 20
4= Guardian: 15
4= Independent on Sunday: 15
6= The Sun: 10 (average of 15, 5)
6= Mail on Sunday: 10
6= Mirror: 10
6= Daily Telegraph: 10
10= Sunday Telegraph: 5
10= Sunday Times: 5
People newspaper: data not available

The scoring works as follows:

  • Has the poll been commissioned from a company that is following the British Polling Council (BPC) rules? 5 points
  • Has the newspaper provided all the information required under C 2.1 of the BPC’s rules? 5 points (with a broad interpretation, e.g. saying the poll was done by YouGov counts as also giving that it was an internet poll)
  • Has the newspaper directly or via the pollster provided the voluntary information listed under C 2.2 of the BPC’s rules? 5 points
    (For both this and the previous point no points are scored if the newspaper reports figures from the poll one day and provides the other information the next day.)

[In other words, basic good practice will score 10 points, really good reports will score 15 points and the worst will score 0 so far.]

  • Are the changes in party support from the last poll by the same pollster using the same methodology given? 10 points (or 5 points if other context given that is accurate but no +/- scores for individual party scores)

[Double points here as changes are frequently at the heart of the narrative surrounding the poll, shape the headline and so on, so following – or not – this rule makes a big difference to the impact of the report.]

  • Are changes in party support within the margin of error described as such or, where greater significant is attached to them, is other polling evidence presented to justify placing weight on the changes (e.g. three polls in a row from different pollsters all showing Labour support up by two points justifies a conclusion that the one poll wouldn’t)? 5 points (also awarded if no changes reported)

[Note: the margin of error is more like 5% than 3% (see footnote here) when comparing different polls  so the “other polling evidence” point should often be used.]

  • Are any other numerical claims made about the polling results which are not actually supported by the poll or other similar mistakes made in the reporting of the poll? -5 points for each one
Read more by or more about , or .
This entry was posted in Polls.

Post a Comment

Lib Dem Voice welcomes comments from everyone but we ask you to be polite, to be on topic and to be who you say you are. You can read our comments policy in full here. Please respect it and all readers of the site.

If you are a member of the party, you can have the Lib Dem Logo appear next to your comments to show this. You must be registered for our forum and can then login on this public site with the same username and password.

To have your photo next to your comment please signup your email address with Gravatar.

Your email is never published. Required fields are marked *

Please complete the name of this site, Liberal Democrat ...?


Recent Comments

  • User AvatarJoseph Bourke 17th Jan - 1:45am
    Expats, I think I have been clear that I support Libdem housing policy as updated last spring This policy includes allowing receipts from the...
  • User AvatarMartin 16th Jan - 11:48pm
    Vince and our other parliamentarians need to remind Mrs May that her self imposed red lines are the stumbling block. We should not promise much...
  • User AvatarYeovil Yokel 16th Jan - 11:43pm
    “What should Vince say to Theresa?” - my reply would be unrepeatable, but my advice to Vince (not that he really needs it) is not...
  • User AvatarDavid Becket 16th Jan - 11:18pm
    It was a fair point from Gove, there was a tme when we favoured a referendum. To many we give the appearance of inconsistency on...
  • User AvatarJayne Mansfield 16th Jan - 11:11pm
    @david Warren, I must eat my words when I say that I do not detect a groundswell for a change in voting system. There was...
  • User AvatarFiona 16th Jan - 11:09pm
    David, no tying in knots required to distinguish between a vote to validate the details of a massive constitutional change and a vote to instigate...