Cross-posted from The Wardman Wire:

Both the recent controversies over whether or not general election counts should take place on the Thursday night and whether or not the 2012 London Mayor and Assembly elections should use e-counting touch, in part, on the question of the accuracy of manual counts.

This is an area where systematic evidence is very thin on the ground. For example, when recounts take place, there is no formal recording of the different recount results nor, when a result is declared, a formal recording that a recount took place. As a result, systematic analysis such as how often recounts occur, in what sorts of counts, and so on is all impossible. (Note to Electoral Commission: gathering this sort of data would probably have more practical benefits than some of the highly detailed but incomplete financial data you have got into collecting.)

As a result, comments such as “Thursday counting is bad because it means tired people and they’re more likely to make mistakes” can’t easily be tested against evidence other than people’s personal experiences.

So here are my experiences of how and why counting goes wrong in election counts with manual counting and either first past the post or regional PR.

First, though, you may quite reasonably ask why my personal experiences should carry any weight? The answer to that necessarily involves some self-puffery, for which apologies. I’ve been regularly going to election counts for two decades, averaging several counts a year and across different Returning Officers each year. Add in to that often being on the phone to someone in a count elsewhere in the country giving them advice, and I’ve got direct experience of a larger, more diverse (i.e. not just the one council) set of election counts than nearly anyone else. There are a few people with more experience, but not many. In addition, I’ve co-authored several editions of two election law manuals (for the Liberal Democrats) and served on the Electoral Commission’s Political Parties Panel for several years.

Back to the substance: how do counts go wrong?

The first stage in a count is emptying each ballot box and counting the number of ballot papers in that ballot box. This is then checked (“verified”) against the record of the total number of ballot papers that were given to voters to complete and place in that ballot box. If ballot papers given out equals ballot papers returns then you know the ballot box has not been stuffed. The numbers should tally, though quite frequently the numbers are out by 1 or 2 for individual ballot boxes. The variation is usually due to a combination of the occasional ballot paper being taken away by a voter rather than being placed in the ballot box and the occasional error in recording the number of ballot papers issued.

The verification stage is fairly robust because it involves checking data from two places (papers in ballot box, record of papers issued). It would be a remarkable coincidence for both to be wrong – and in the same direct and by (almost) the same amount, so that cross-checking the two totals doesn’t bring that to light.

In practice, the two items which can go wrong at this stage are sloppy counting and missing boxes. If sloppy counting is combined with a Returning Officer who is happy to let the verification stage pass even with discrepancies greater than 1 or 2 per ballot box, the result is totals which may well not tally with the final vote counts. If that happens, it’s easy then to be stuck with no way of finding out what went wrong and where in the process. Missing ballot boxes happen only rarely, but just sometimes a box is misplaced or forgotten and not included in a count. One example is a ballot box that was placed under a table and then forgotten about for the rest of the count.

In both cases, the safeguard should be a simple and clear system for recording the totals of ballot papers issued and counted in each ballot box, with a line for each ballot box so you can see if any are missing altogether. In addition, some Returning Officers show these calculations to election agents – which acts as an extra protection against mistakes.

In other words – the main cause of problems caused at this stage are poor systems (e.g. no check that all the ballot boxes have been included) or a Returning Officer who sees sharing information with election agents as a burden rather than as an extra insurance against errors. A tired person, with a good system will perform better than an alert person with a bad system.

The next stage is to then mix together ballot papers from different ballot boxes and to sort them out in to piles for each candidate. It’s at this stage that postal ballot papers are added in too. They will have gone through a slightly different process up to this point with again the main risk being that a large number of papers are mislaid and then main protection being good systems.

The main causes of mistakes during the sorting in to piles are either individual papers being placed in the wrong pile or, when a pile is scooped up to be placed with other votes for the same candidate as a prelude to counting them, placing a pile of votes for Candidate X in with votes for Candidate Y.

This is where the counting agents for each candidate can provide an important safeguard – as they will be watching out to make sure that none of their own votes are misplaced in this manner. Again, the attitude of the Returning Officer is crucial to how effective this safety net is. Did they allow a reasonable number of counting agents such that they can properly scrutinise the count? Are they given reasonable space to use and allowed to get close enough to the counting to see what is being done? Or are counting agents a headache to be kept away and is shifting around bits of paper out of sight under table tops seen as acceptable? And most crucially – are the bigger piles that the papers are gathered up in to in clear sight of the counting agents, who can therefore spot large pile of votes going astray?

The next stage is then to take the ballot papers for one candidate and start counting them up, usually in to batches of 25 or 50. The batches are then piled up and in turn counted up in order to get a vote total.

As in the previous stage, the main source of error is batches going in the wrong place. Take a batch of 50 votes, accurately counted, but put it in the wrong candidate’s pile and the accurate counting was for naught. As before, a major safeguard is the role of the counting agents. The other main safeguard, as all through the process, is the quality of the supervision of the counters by the “middle management” at the count. Flicking through piles of ballots, taking a second glance before moving bundles around and so on – that can all make a big difference.

For multimember elections, such as when three ward councillors are all up for election at the same time, this counting stage is a little more complicated because there are multiple votes on the same ballot paper. Often therefore the “block votes” where each vote is for the same party are sorted out, and then tallies made for the split party votes. Alternatively, tallies may be made straight-away. Tallying can either be done by recording five-bar gates as you go through each ballot paper in turn or by laying out a large number of ballot papers next to each other and then counting across to total the total number of votes for each candidate in that batch.

Whatever method is used, this is a stage liable to errors – and the sort of errors that it is hard for others to spot and query. However, as with other stages large errors are almost certainly going to result from either mathematical mistakes or from a batch of papers being put in the wrong place.

After the counting, the bundles (or tallies) of votes are added up for each candidate. The total of votes for all candidates, plus rejected ballot papers, should also match with the total of all the ballot box verifications.

Mathematical errors at this stage can make a large difference to the result, and this maths is often done away from agents and without them being given the sums to check and query. The systems used also vary greatly in quality, with some Returning Officers having spreadsheets into which all the numbers are plugged and any failures for totals to cross-check automatically flagged. Others do no cross-checking.

In other words, the main errors at this stage come from poor systems and from not allowing agents to cross-check calculations.

There is a common theme running through the stages. It is true that at all stages the more tired people are, the more likely mistakes are to happen. But in reality the far bigger impact on accuracy is caused by the systems used for the count. Clear records for collecting, totalling and cross-checking numbers protect from a myriad of possible errors. Sums that are only done once, not cross-checked and only dashed out on a scrappy piece of paper are far more vulnerable to error.

In my experience of mistakes being made at counts, there is a near inevitable low level of errors with individual papers going in the wrong pile or a pile of 25 papers actually being 26. These sorts of errors can be related to how tired or fresh the staff are. But the big errors – the ones that can change results – are of a different sort. They are the ones where poor systems allow large number of voters to be missed out, misplaced or miscounted.

Returning to the specific issue of whether counting with staff on a Thursday night means mistakes are more likely to happen, that is missing the main issue. If you’re worried about accuracy (and given what some Returning Officers get up to, there are times I certainly have been) then you need to worry about systems. Worrying about Thursday night versus Friday morning is like worrying about whether you are going to start a car trip in Reading or Oxford – but your destination is Beijing. It’s a trivial difference to the overall issue.

If you want a practical suggestion for reassuring people that election counts are accurate, don’t worry about Thursday night versus Friday morning. Instead introduce a new rule requiring the verification and vote tally calculations to be published along with the count result. That will force those Returning Officers with poor counting systems to raise their game – and that’s what really matters.