Media to start getting marked for quality of opinion poll reporting

The quality of traditional media coverage of political opinion polling has been a common cause of complaints amongst political bloggers. The most obvious problem is when an opinion poll from one polling company is compared not with the previous poll from that company but against an older one because the intervening one happened to have been published by a different media outlet.

Whilst comparing, say, the latest ICM poll with the previous ICM poll is the most useful comparison to make, if that previous ICM poll appeared elsewhere, in the part it has got airbrushed out of report of the latest one. The weirdest example of this was when The Independent and Independent on Sunday alternated publishing polls from the same pollster, but each didn’t mention the intervening poll in their sister paper when reporting their own.

Some of these failings are becoming less frequent, and in particular The Guardian now much more frequently makes references to non-Guardian polls when trying to make sense of its latest polls. Although The Telegraph recently messed up comparing a YouGov poll with the most recent relevant one, that was due to an oversight by the journalist rather than a deliberate decision.

So there is progress, helped no doubt by the criticism from Anthony Wells and Mike Smithson, both of whom are respected by many of the relevant journalists.

However, there is still much more that could be done to raise the overall quality of such reporting, so here at The Voice we’re going to start scoring each poll commissioned by a traditional media outlet and the way in which its initial report is worded.

Once the scores have been running for a little while, we can have the fun of a league table showing the best/worst journalist/media outlet at reporting opinion polls. Fun – but also a way to encourage those who want to meet high standards to raise their game.

But all that requires a sensible marking system.

First thoughts on factors to include:

  • Has the firm been commissioned from a company that is a member of British Polling Council (BPC)?
  • Has the poll followed the BPC’s rules?
  • Does the newspaper report give the fieldwork dates for the poll?
  • Are the changes in party support from the last poll by the same pollster using the same methodology given?
  • Are changes in party support within the margin of error described as such or, where greater significant is attached to them, is other polling evidence presented to justify placing weight on the changes (e.g. three polls in a row from different pollsters all showing Labour support up by two points justifies a conclusion that the one poll wouldn’t)?

Once we’ve drawn up our marking system, we’ll backdate it to start with the first poll of this year and then publish figures regularly through the year. If you’ve got any views on what should or shouldn’t be included, pop up a comment.

Read more by or more about , , , or .
This entry was posted in Polls.
Advert

One Comment

  • The Sun leads on another ( massive ) Yougov poll today but on examining the detailed data tables :-

    1 ) This seems to be an amalgam of their 2 polls last week plus some extra responses , The fieldwork all took place from 5th to 7th Jan .
    2 ) The headline figures given in the Sun 42/30/16 are wrong , The data tables show 40/30/17
    3 ) The sample is very heavily biased with responses from those over 55 especially males ( 45 % over sample of males over 55 for example )
    4 ) The sample undersample those with a LibDem ID by 20% 1005 compared to weighted 1242 but a correction for this does not seem to occur in the final weighted figures – the LibDem replies only increasing by 2% from 1357 to 1385 .
    It would seem that the LibDems need to be very vigilant over the reporting of polls over the next few months and respond with detailed criticism where as with this Sun poll they are clearly reporting incorrect figures on a poll with dubious data .

2 Trackbacks

Post a Comment

Lib Dem Voice welcomes comments from everyone but we ask you to be polite, to be on topic and to be who you say you are. You can read our comments policy in full here. Please respect it and all readers of the site.

If you are a member of the party, you can have the Lib Dem Logo appear next to your comments to show this. You must be registered for our forum and can then login on this public site with the same username and password.

To have your photo next to your comment please signup your email address with Gravatar.

Your email is never published. Required fields are marked *

*
*
Please complete the name of this site, Liberal Democrat ...?

Advert



Recent Comments

  • User AvatarJoseph Bourke 17th Jan - 1:45am
    Expats, I think I have been clear that I support Libdem housing policy as updated last spring https://www.libdems.org.uk/spring-18-f4-local-government-housing This policy includes allowing receipts from the...
  • User AvatarMartin 16th Jan - 11:48pm
    Vince and our other parliamentarians need to remind Mrs May that her self imposed red lines are the stumbling block. We should not promise much...
  • User AvatarYeovil Yokel 16th Jan - 11:43pm
    “What should Vince say to Theresa?” - my reply would be unrepeatable, but my advice to Vince (not that he really needs it) is not...
  • User AvatarDavid Becket 16th Jan - 11:18pm
    It was a fair point from Gove, there was a tme when we favoured a referendum. To many we give the appearance of inconsistency on...
  • User AvatarJayne Mansfield 16th Jan - 11:11pm
    @david Warren, I must eat my words when I say that I do not detect a groundswell for a change in voting system. There was...
  • User AvatarFiona 16th Jan - 11:09pm
    David, no tying in knots required to distinguish between a vote to validate the details of a massive constitutional change and a vote to instigate...