Nate Silver grabbed the headlines last year when he correctly predicted the outcome of the US presidential election in all 50 states when other commentators were expecting a dead heat. Since then, he has sold his fivethirtyeight blog to ESPN and will continue to edit it, expanding it to become, he hopes, the “go to point for data driven journalists.” It will also give him the chance to do political punditry for ABC News.
He came to the Edinburgh Book Festival earlier this week to talk about his book The Signal and the Noise: The art and science of prediction. The headlines were full of his prediction that Scotland was virtually certain to vote to stay in the UK. I wonder if the comments on that article will make him revise his opinion that people in the UK are more sane and rational when discussing politics than they are in the US. He said that pro independence blogs tended to be more rational and considered than Republican equivalents and in most cases I’d agree with that – although I’d point out that he’s not setting the bar very high.
He elaborated on the points he made in that Scotsman interview, saying that polls were more likely to turn in an election than on a matter of public importance like a referendum where the question was fairly straightforward and known well in advance. The polls have been relatively stable since that question was announced.
He was asked where he found out his news. His first port of call is Twitter, followed by conservative and liberal sites along with the New York Times and the Guardian. He praised the latter’s coverage of US politics.
The internet, he argued, should make people more accountable. In theory, it should lead to greater consensus as the wide variety of information should lead to better debate and consensus. Sadly, though, because people filter what they read according to their beliefs, it encourages greater partisanship.
He’s self-assured in person, certainly, but more engaging than I had expected. He talked about the difference between writing his book and the blog. A book gives you the chance to develop a coherent argument, “you get to hold serve for a while”, while a blog is more conversational. He has great confidence in his predictions which is hardly surprising given his record in presidential polls. He talked about the North Dakota race where the Democrat triumphed despite having only an 8% chance of winning. As Silver says, those 1 in 10 probabilities come up around 1 in 10 times.
Asked about the value forecasting, he said that in science it’s very important to be able to predict what’s going to happen in a situation. Climate change came up a bit later. He said that scientists tend not to get involved in the political streetfight and it’s not easy to simplify the data anyway. Regional impact is more difficult to predict with certainty. He warned against blaming every storm on climate change.
Politics is a much more fluid environment, though and there are many other factors to take into account. He mentioned the “shy Tories” factor in England in the 1990s – where they did better than expected because they were reluctant to admit their vote to pollsters. This is not a phenomenon that’s seen in the US. He talked about the art involved in building a program to analyse the data, weighting it to give a consistent and accurate picture of what’s going on.
He’ll start the data collection for the 2o16 presidential election in the Spring, but information after the campaign starts properly after Labor Day is most useful In the early stages of a campaign, data on the economy can also be useful in making predictions – although he said that economic forecasters make political forecasters look good because it’s harder to measure all aspects of economic data. He gave as an example the housing booms post war and in the 2000s. In the former example, there was a huge demographic shift after the war which made house prices rise. In the 2000s, the rise was driven by speculation and we know how that ended up.
Nobody asked him what he thought would happen in the next British General Election which I can’t imagine happening south of the Border. The referendum still seems to be the only game in town up here, although, if Silver is right, the 2015 general election will have as much relevance to Scots as everyone else.
* Caron Lindsay is Co-Editor of Liberal Democrat Voice and blogs at Caron's Musings