In this post: Exam Results and Gradings Ianto Stevens explained how and why examiners moderate exam results to try to smooth out inconsistencies from year to year. Some of this was to compensate for variations in the difficulty of exam papers. The overall aim was to give the same balance of grades as in previous years.
For example, suppose on a particular A Level physics paper, students across the country get markedly higher marks than in previous years. Examiners will assume that the students are of much the same ability as previous cohorts and that the variations are due to the questions asked and the marking guidelines. It is almost impossible to write a series of exam papers that produce the same range of results each time.
The problem this year was that this legitimate moderation process was applied not to actual written exam papers but to teacher’s predictions.
I spent many years teaching and organising A level and BTEC courses in a large FE college. Each year the whole of my Easter break was devoted to assessing and ranking coursework projects for some 60 A level Computing students. And, of course, I had to provide predicted grades which I based on their AS levels, mock exams and the state of their coursework at the time when the predictions had to be submitted.
I do understand why there was a reluctance to simply award the students with the grades predicted by their teachers. If that had happened then the overall grades would have been significantly higher than in previous years. The consequence would have been that a higher number of students would have met the conditions set by universities, so there was a danger that courses might have been oversubscribed. Universities always offer more places than they can fill, on the basis that not all will qualify.
So, to avoid a glut of qualified students, the raw predicted grades were treated as though they were actual marks and were moderated to bring them in line with other years. What is more, the moderation was applied at the level of an individual school or college rather than across the whole exam entry cohort – a granular application of a holistic method.
I will argue that it was not necessary to moderate the grades at all, but first let’s take a look at the procedure that was adopted. The actual predicted grades were ignored and the rankings used instead. These were mapped on to the spread of grades achieved in that subject, in that school or college, over several previous years. So if in previous years, on average, 5% had been awarded a U (ie fail), then the bottom 5% in the ranked list this year would be given a U, and so on across all the grades.
You can immediately see how this could produce some serious anomalies. Suppose this year’s teaching group was especially good – or were being taught by a better teacher – and were all predicted to get a grade C or above. Under the moderation algorithm at least one of them will be awarded a U.
To make things worse, the moderation process was not used if a school entered fewer than 15 students in a subject, or if it was a new subject with no past results for comparison. Again, on the face of it this may seem reasonable, given that small classes can vary significantly in quality from year to year. In these cases the teachers’ predictions were used as the final grade.
But of course, as we all now know, that only produced even greater unfairness, because only independent schools can afford to run small classes. Thus students in private education were much more likely to be awarded the grade their teacher had predicted than students in large comprehensives or FE colleges, who took the brunt of the downgrading.
It appears that when this year’s moderation process was being developed the unintended consequences were simply ignored. Where have we heard that before?
We have also seen that the controversy has been handled differently in the four nations of the UK. In Wales, Education Minister Kirsty Williams has announced that no student will receive a final grade lower than the results of their AS level in the subject. In England that can’t be done, because Michael Gove abolished AS levels as a stepping stone towards A level, and he also abolished coursework in many subjects – all against advice from the teaching professionals.
So what do I think should have been done this year? To start with the Government should have acknowledged that A level students have had a really rough ride. This year is an exceptional circumstance and should not be used to justify outcomes in future years.
Would it have been so disastrous to award the teachers’ predicted grades? After all, in England at least, there is literally nothing else to go on. I don’t think the universities would have been overwhelmed as they feared. Many are losing overseas students this year because of travel restrictions. Also quite a number of the current 18 year olds are considering taking a year out because they don’t want to lose the full campus experience and would prefer to wait until teaching is all delivered in person. Maybe fewer students would have ended up in clearing but that is not a bad thing either.
The Government, and Ofqual, have had plenty of time to seek advice on the fairest way to deal with this problem. I guess we will never know the conversations that went on behind closed doors, but someone has made an almighty mistake.
* Mary Reid is a contributing editor on Lib Dem Voice. She was a councillor in Kingston upon Thames, where she is still very active with the local party, and is the Hon President of Kingston Lib Dems.
Follow @LibDemVoice on Twitter
Like us on Facebook
Subscribe to our feed
Sign-up for our daily email digest
Satisfaction with the NHS falls to lowest level since records began in 1983
Ed Davey launches Lib Dem local elections campaign in Berkhamsted
My tweets
UK Government to reintroduce Victorian prison hulks?
Roads and pavements improvements for the West End 2023/24 #dundeewestend
Shropshire Council Liberal Democrats call for dedicated active travel budget
Steve Trevethan
Mel Borthwaite
Nonconformistradical
Roland
Jenny Barnes
25 Comments
Yes, Mary. This has indeed been an exceptional year. The easiest way to resolve the problem of grades as far as England is concerned is probably to go with teacher assessments. Mind you, I still reckon that it would not have been impossible actually to have held the exams with a bit of ingenuity, or at least some form of test taking into account missed teaching time after March.
Let’s just hope that this year is indeed a ‘one off’, although we cannot be sure.
You have to laugh, one moment mocks will be used in the triple lock, the next thing you know they will only be used under certain, quite restrictive circumstances. And then that idea was quickly withdrawn.
I agree, the students should have been given the benefit of the doubt – after all, with much more unemployment it is in everyone’s interest to keep them in education.
I am surprised the LibDems have not taken the moment to suggest that uni education should revert to being free for the poorer students for the next couple of years, though, or even the whole lot, that might get the youth vote back on side.
Yes, from me too, Mary.
But cutting to the chase, what is Ms Moran’s policy towards Academy Schools and whether they should be returned to LEA Control ?
The whole culture of Academies is questionable with talk of early warnings about Ofsted visits, absurdly high salaries and honours dished out, and the former founder of a Chain of Trusts now holding Ministerial Office in the DFE.
Maybe John can fill us out on the Lincolnshire experience.
Thanks for the invitation, David. Since its inception in 1974 Lincolnshire County Council has been completely Conservative controlled except for two periods, 1993 to 1997, when it was run by a Lib Lab administration and 2013 to 2017 when it was run by a coalition of Tories, Independents and Lib Dems, of which yours truly was a member.
It retained selection at secondary level in most parts except for Greater Lincoln, closed down Primary School kitchens in the early 1980s, encouraged the establishment of Grant Maintained Schools in the early 1990s and their conversion to Foundation Schools at the end of that decade, and then turned most of its poor performing secondary modern schools into Academies – and a good number of its Primary Schools as well.
In the early 2000s it also turned a newly created Science and Technology College in Lincoln into a de facto Grammar School by allowing it to cherry pick the best students from the Greater Lincoln area. It had tried something similar In the run up to the 1993 County Council elections with the only comprehensive it controlled in Greater Lincoln. Unfortunately it hadn’t taken into account that all the other comprehensives had, by becoming Grant Maintained, not only opted out of its control; but were able to field ‘Say No to Secondary Moderns’ candidates against sitting Tories. That’s how we gained one of the seats in my patch, which we held until 2017. The incoming Lib Lab administration of course reversed the decision. When they returned to power in 1997 the Tories got their revenge, as I indicated earlier, by the back door.
However, that wasn’t the end of the story. The Executive Head of the little empire that emerged as an Academy Chain from the Science and Technology College, was prosecuted a few years ago on various counts, including the funding of an apartment in France, trips abroad, giving a job to his son, who was already a convicted sex offender, and the purchasing of ‘sex toys’. Although acquitted, his reputation was trashed and he departed. I gather that he might still be teaching……in Scotland!
@John Marriott – fascinating!
Just been chatting with a friend who has a senior role in one of the Russell Group universities. He is worried about admitting too many students as they would be penalised to the tune of several million. In my view the Government should drop the strict limits on places for one year only and put it down to Covid-19 expenditure.
Thanks for this very clear explanation of how the shambles has happened. What has to be added of course is the way this Government responds to more and more crises by back of fag packet instant solutions announced at very short notice (or none at all). It is all chaos and disruption – the creation of which is of course a hallmark of Mr Cummings’ basic beliefs. Combined with a set of utterly mediocre ministers, the pattern is becoming clear.
It’s going to get even worse on Thursday when the GCSE’s come out.
@David Raw – yes, the scale of the problem with GCSEs will be even greater, but the consequences may not be quite so serious. It is possible to recover from poor GCSE grades but much more difficult to move on from unexpectedly poor A Level grades.
Mary Reid:
There is a future problem though with GCSEs. Universities often rely on GCSE records for deciding offers. Unless there Is a rapid decision to reintroduce intermediate exams, as there is in Wales, there will be no reliable externally validated data. I think that exam boards should be required to make an optional intermediate exam available next Summer for all who were unable to sit GCSE exams.
@Martin “I think that exam boards should be required to make an optional intermediate exam available next Summer for all who were unable to sit GCSE exams.”
The new style AS-levels are already available for that, and although not as integrated into the A-level syllabus as their predecessors, they are intended to be co-teachable (if that is a real word!). However, the Coalition government’s reforms to A-levels were intended to remove the disruption of third-term exams and allow for 5 terms of teaching so additional exams at the end of year 12, probably coupled with time for extra remedial/catch-up teaching at the beginning of year 12, could introduce other problems for pupils and teachers.
“Would it have been so disastrous to award the teachers’ predicted grades?”
I don’t think there is any solution that is fair to everybody. After all, this year’s students will be competing for jobs or places with students who had exams last year or next year, but this year’s cohort could have inflated grades and not had the benefit of intensive revision and testing to consolidate their learning.
However, one argument for using the teachers predictions of what their pupils are capable of achieving is that, if they are based upon a rigorous and honest approach, in a normal year one might expect far more students to mess up on the day of the exam and underperform than to miraculously improve so the actual results would always be worse over all than the predictions. But there is no way to predict who those unfortunate people would be and this year’s moderation process seems to spread that bad luck across everybody but in an unfair and disproportionate way.
Half of A level students don’t go to university. Such students go directly into employment. Employers will want A level grades to be at a similar level to previous years and not at the highly inflated level of some teacher predictions for their students.
Sadly some teachers will try to game the system and deliberately inflate predicted grades because it will boost their own apparent performance.
Having seen what’s happened with A levels our government aka Boris Johnson and Dominic Cummings has three days to come up with a foolproof system to prevent chaos again. How about offering teacher assessment or algorithm, which ever is the higher? Anyone got a fag packet they could let the poor blighters have?
Thank you, Mary Reid, for a really clear and thoughtful contribution.
I am very anxious that the whole of our national system for assessing the attainment, ability and potential of students be carefully looked at. The weaknesses in it are partly down to Mr Gove but they have been there from before his time and have been revealed by Covid.
Is there a LibDem policy group working in this area? If not, one should be convened asp.
A former Planning Engineer writes:
The most dismaying aspect of this is knowing that what is a major problem for universities and students from now to October and beyond was clear once lockdown started and exams were cancelled. When was that? March? April? A fully working government had plenty of time to gather a project team of experienced people to work out a much better way of doing it, even with the handicap of Gove’s abolition of S-levels.
The motto of this government seems to be ‘It’ll be all right on the night.’ ( (c) D.Norden)
@Martin – In most sensible countries there are no public exams at 16 and students apply to university after they get their results. It’s much simpler and more humane all round. Of course, that doesn’t solve this year’s fiasco.
Oh, and I meant to add, give Williamson his P45. As a reader wrote in today’s Guardian, politicians are like nappies; “Both need to be changed from time to time, and for much the same reason”.
A teacher friend gave the analogy that had the Premier League been abandoned this year, with Liverpool FC miles ahead, an algorithm could have been used which said “Liverpool, your normal position is around 4th so we’ll say you’re 4th this year”.
More seriously, this same teacher had a class massively downgraded (by virtue of poor area, other subjects not being rationalised with too few pupils etc etc) and one student who she predicted would be a B or C (but the actual teacher prediction, erring on the side of caution was a C), who needed a C to do the course she was to do, was given a U. Then you have Gavin Williamson praising Eton for its best ever A-level grades and you have to feel something isn’t quite right.
@ Julian Tisi “Gavin Williamson praising Eton for its best ever A-level grades and you have to feel something isn’t quite right.”
Fact is, whenever Boris took off his white helmet and viz jacket to disappeared again, he was actually sitting in Downing Street devising a hand crafted algorithm especially for Eton.
@David Raw
I don’t credit him with that much intelligence.
If there is no alternative to using teachers’ predicted grades then efforts should have been made to ensure these are as accurate as possible. Teachers will rightly think the best of each student. With some homework of their own around bias teachers might have delivered a more realistic and fair assessment of each student.
My question is what happens next? It appears increasingly likely that England follows Scotland’s U-turn but where does that leave students in Northern Ireland and Wales? While the backup is more robust in Wales, the same system has been used with the same impact of where they live/which schools they attend resulting in 42% of grades having been lowered, initially.
Lewis Goodall, Newsnight, twitter describes speaking to a Vice Chancellor of a University who says part of the reason they cannot be flexible is that now all Scottish students have met their offers (I have seen other tweets to suggest Universities are holding places if appeals can be completed September the 7th) but if English students receive a u-turn then this issue just gets shouldered by Welsh and Northen Irish students only? We surely then see same u-turn in Wales and Northern Ireland but still with extra delay and Universities becoming less flexible as they become fuller.
Mary Reid:
It were better to annul GCSE grades this year, simply giving pass or fail alongside the teachers’ projected grades. The situation so far as future university applications is concerned will be made worse by allocating unsubstantiated grades.
Indeed elsewhere it is the norm to continue with core subjects (Maths, 1st and 2nd language, perhaps Philosophy) alongside specialist study. Universities still look at students’ past record and teacher projections.
@Martin – Unless there Is a rapid decision to reintroduce intermediate exams, as there is in Wales, there will be no reliable externally validated data.
I agree this is the fundamental problem we have, there is no universal system in place for continuous assessment and thus giving confidence in grade allocations.
Given CoVid19 is going to be with us for sometime, it does seem sensible that moving forward the urgent need is for some preparation to be made, if only so that we don’t get a repeat for the 2021 GCSE and A-level’s. Remember this years GCSE/A-level students only missed about a month of education, next years will have missed 4 months, so there will be much less information on which to make assessments…
Aside: Yes I have a vested interest as have children sitting both GCSE and A-levels in 2021.
Update: Wales has come to decision to use centre assessment grades across the board.
The message coming out of this has to be that Wales had more robust system and immediately made appeals free of charge, however due to changes in Scotland and expected changes in England that alterations had to be made to ensure fairness for younger people. While there is some issue in Wales (predominantly it being such a late decision) the issue largely lies in Scotland and England who have therefore had to over-correct putting Welsh students at risk of living with comparatively unfair outcome.
What now of Northern Ireland though? Are they next to change approach as they currently have, in comparison to other nations, under-corrected their approach?