On day zero we heard of some horrendous downgrades. Perhaps that was just anecdote. No longer. Clearly now we are in the realm of overwhelming evidence that the awarded grades are not a fair indication. Every single FE college responding to a survey covering half of all colleges report lower grades than in previous years. Large numbers of independent schools are bragging about record high grades. The stolen A*s have been awarded to others and the top university places have therefore largely been taken.
In principle, an algorithm could be fair from one school to the next, giving effectively each school its prior attainment. Clearly the algorithm hasn’t done this and therefore clearly it contains errors.
Even without these errors, the concept that if your centre expects a U to be awarded, then somebody has to be given a U, however well they are actually performing, is grotesque. But this is the concept, and it applies to all the grades below the level you are performing at, not just the Us.
The idea that the formula doesn’t work for small cohorts is correct. The remedy of granting inflated grades to students in small cohorts (largely public schools) and deflated grades to everybody else, is worse than the disease. The formula does not work for large cohorts either.
This brings me to the final conceptual error in the whole enterprise. Had the formula been correct, and had a fair solution been found to the problem of small cohorts, we would have been in a position where each school got broadly the results they would normally get, and we might all shrug and say ‘well that seemed to work’. Universities would have right number of applicants meeting their offers and would be happy.
But even then there would be no fairness from one individual to the next. The hard working late bloomer would be beaten by the coaster every time. There would be disappointed students, but we expect that anyway. There would be no proof a student could offer that they had mastered their subject. We would have fixed the aggregate statistics to look fair, and betrayed the individual students. The unfairness this fiasco has uncovered is only the tip of the unfairness iceberg.
How was this ever thought to be acceptable? I think the malign influence of collectivism has caused us to think of fairness in aggregate terms rather than individual terms: if somebody else from your school gets your grade, that doesn’t matter because they are the same as you, as far as the stats are concerned.
But the only fairness that matters is fairness to individuals.
Now that uni places have already been offered, I think the only remedy is to revert to CAGs (school assessed grades) – as Kirsty Williams has announced will happen in Wales – and to fund an expansion in places. Universities that have taken the invented grades at face value may deserve a bloody nose but it is more important to fund the extra places anyway to remedy the injustice to individual students. Universities should have capacity for extra places as they are losing international students.
* Joe Otten was the candidate for Sheffield Heeley in June 2017 and Doncaster North in December 2019 and is a councillor in Sheffield.
Follow @LibDemVoice on Twitter
Like us on Facebook
Subscribe to our feed
Sign-up for our daily email digest
Getting things done - Colinton Place #dundeewestend
Have cyber criminals been swindling taxpayers?
Back to Butterley Tunnel on the Cromford Canal
Are we programmed to die?
Steve Trevethan
Alex Hosking
graham
Daniel Stylianou
Craig Levene
25 Comments
They’ve announced that CAGs will be used in England.
It isn’t clear what this means for university places that have already been lost.
Funny how it just happened to hurt the poor the most. Having a kids whole schooling hanging on a computer is just not on really and I have no idea why this was even tried.
For years we have been using the exam system as a means of and a pretext for denying young people the chance to follow their dreams. There will never be a better time than now to tear up the whole thing and start again.
“But the only fairness that matters is fairness to individuals.”
I love this line!
There are conceptual errors associated with relying on CAGs as well. While many, perhaps most, teachers have given their students accurate grades; some have obviously been wearing rose-tinted spectacles otherwise we would not have seen the grade inflation that led to the downgrades in the first place. Some students will not go on to university and for them, those predicted results will become the data that their potential employers rely on to choose between similarly-qualified applicants. Do they deserve a bloody nose as well? At least the students that go on to university will have another opportunity in three years’ time to demonstrate their qualities.
@Chris Cory “For years we have been using the exam system … There will never be a better time than now to tear up the whole thing and start again.”
One of the things that upset me most about the Lib Dems in Coalition is that they seemed to align themselves ( once I would have said we aligned ourselves, sigh đ ) too closely with Michael Gove’s reforms and an increased emphasis on terminal exams at 16 & 18. It makes some of the criticisms this week ring hollow, but as you point out, there is probably an opportunity now to push for radical change if only someone can be clear about what that change should be.
But we’ll probably just get a review and then a recommendation that teachers’ administrative burden is increased by carrying out the rigorous prediction exercise every year alongside the exams.
Ofqual/Department for education are guilty of an innumerate abuse of statistics.
Whilst on a statistical basis it is justified to expect a distribution of grades, it is entirely unjustified to attribute specific grades to particular students on that basis. Yes, in a class of 18 a U grade might be expected, but provides no basis to whom a fail grade applies. Much the same applies to all grades, but particularly to outliers.
The government has now created a new mess in terms of University placements, they have created some hugely oversubscribed courses and, because of the UCAS choice system, some unexpectedly undersubscribed courses for which the universities could not possibly have planned for, nor even anticipated. There will be multiple problems.
Excellent post, Joe. And I am so pleased that Ofqual has done a U-turn, although, as you imply, that still leaves a problem for universities that may have to honour more offers than they were expecting. Under the current rules they actually get fined if they take more than their quota.
@Laurence Cox – yes, indeed, there are some issues with teacher assessments including some deliberate overestimates in some schools. But remember that this year there was nothing else to base the final grade on. But teacher assessments are overall a much fairer way of doing things than any of the possible alternatives.
@Mary Reid “But teacher assessments are overall a much fairer way of doing things than any of the possible alternatives.”
That’s the crux of it. I don’t think there is any approach that is absolutely fair to all interested parties, including past and future pupils.
I’ve little sympathy for past, present and future Tory Education Secretaries, but I’m also unimpressed by the demonisation of Gavin Williamson in some quarters. Like Kirsty Williams, John Swinney, and Peter Weir, he’s had to deal with an unprecedented situation (of which year 11 & 13 school exams are only a part, albeit an important one) and at the same time they and the employees of every organisation involved (like many of us!) have had to cope with novel working practices. I really don’t envy any of them!
Williamson is claiming that he did not see the data and the methodology until “this weekend”! Is he lying or confessing incompetence? It is hard to tell.
Whatever it is utterly extraordinary given there have been concerns ever since exams were cancelled. I suspect it is a ham fisted attempt to shove all the blame on Ofqual, who indeed have been incompetent, but it has been Williamson’s job to forestall incompetence. My guess is that Williamson set impossible parameters and is a long way out of his depth.
Now that they are going to use teacher assessments, is it a case of the triumph of man over the machine?
Thx all. So I understand the cap on numbers has been lifted, so universities will be paid for extra places if they expand courses. Though there may be logistical barriers in many cases.
Clearly the loss of AS levels has added to the fragility of the system in England, as has the need for universities to guess how many high grades will be achieved (a guess justified normally by an assurance from OfQual that the overall results will be reasonably consistent with previous years).
I would favour a system of applications to university post A level results so that applications are more based on evidence and less based on guesswork that the privileged can always game better.
“Universities should have capacity for extra places as they are losing international students.”
Have you ever worked for a university?
There may be fewer international graduate students in Law and Business studies but lecturers in Law and Business Studies can’t suddenly start teaching UK undergraduates in History, Physics and English.
Lecture halls and labs can’t suddenly be built to accommodate larger cohorts in some subjects, nor can staff be readily found to teach them. That’s before you consider the problems of overcrowded library and computer facilities in a Covid world. Courses start in less than a month remember…
Let’s have a grown up discussion rather than just lumbering universities with the problem.
I believe it is the PM’s chief advisor who believes in the reliance data-driven decision making – government by algorithm is right up his street. Now the government has been forced to do what it has been elected to do (and what all democratic governments are elected to do), and use its discretion. Never mind Covid 19, there is a potentially dangerous outbreak of common sense out there.
âBut the only fairness that matters is fairness to individuals.â ABSOLUTELY.
The fundamental error was made right at the start – when Ofqual assumed that its task was the same as in ordinary years – to standardise between years: and the Government did not tell them otherwise. But this year is not an ordinary year – and never can be. It should have been made clear right from the start that the task was to be as fair as possible to the young people unfortunate to be coming to this stage of life in this most extraordinary year.
In Wales, 94% of students would have been covered by one of: i) their teacher’s predicted grade, ii) a higher grade as calculated through the algorithm, or iii) their assessed AS work. The 6% badly treated by this system, and those disagreeing with being marked on their AS work, could appeal for free to ensure fairest possible result….until England and Scotland decided to use teacher predicted grades only which creates unfairness through comparison. The majority of the problem in Wales rests on it being seemingly a last minute decision (surely we could have had results and time to appeal before Universities started to fill up places?) and that AS grades weren’t immediately sent to Universities so places were still lost, however this should not be a case where analysis says all major parties failed as much as each other: Wales’ system was most robust, Scotland handled the PR best and England made the worst decisions and in the worst style.
The problem has shifted now so that Universities and BTEC students are left in a confused and disadvantaged state. This hasn’t finished yet despite, some, politicians attempts to blame the computers and move on.
Joe hits the nail on the head; governments have been building a system increasingly focussed on using tests and statistics to measure school performance, instead of using them to help the learning and progress of people. The move to over reliance on end exams is a fundamental error also and Sam Freedman (former adviser to Michael Gove) admitted that on newsnight last night. We have here the core of a good message to the public in the relevant local elections next May about our local school systems.
I remember the introduction of the AS/A2 system, because my daughter was in the first year of ‘curriculum 2000.’ It struck me as a much betteer system that the old final exam system that was in force when I took A levels in the 1970s.
Yet before the new system had time to properly bed in, along comes Gove and Cummings who rip it up and reert to the old system! As we have seen in Wales if we had retained the old system we would hgave had real marks in real AS exams, and real coursework to base judgement on. An attempt to make A levels broader and more relevant was scrapped because the system rewarded those who worked consistently over two years, rather those who were coached to pass the exam, and ‘crammed’ in the weeks leading up to them.
And we have to ask why Ofqual and or DFE had to wiat until August before releasing the information. In any assessed system there were bound to be appeals, and they should have planned for that.
One of the grosser outcome of this fiasco was post code lottery discrimination.
I am reminded of Tony Benn’s old nostrum : “Those in positions of economic, social and political power should always be asked five questions:
âWhat power have you got?â âWhere did you get it from?â âIn whose interests do you use it?â âTo whom are you accountable?â âHow do we get rid of you?â
Those questions should be asked of the Academy Chains…….. It’s time to return schools to democratically accountable L.E.A.’s., and what is the stance of the Lib Dem Leadership hopefuls on this ?
I see in the Health Sector Boris Chum Dido Harding (of test and trace failure) has been given a nice little earner. The Benn questions should be asked there too.
Dido Harding, a Conservative peer who heads up Englandâs widely criticised test-and-trace system, has been chosen to run a new institute to replace Public Health England, after the controversial decision to axe the agency.
Harding will be named as the chair of the National Institute for Health Protection, which will be charged with preventing future outbreaks of infectious diseases, despite the poor performance of NHS test and trace, which she has led since May.
@Steve Comer “Yet before the new system had time to properly bed in, along comes Gove and Cummings who rip it up and revert to the old system”
I completely agree with the sentiment, but unfortunately Gove & Cummings were ably assisted by a Lib Dem Schools Minister, David Laws.
I recently came across an old post of mine on this in Feb 2013 (https://www.libdemvoice.org/opinion-goves-a-level-reforms-risk-pushing-many-universities-out-of-reach-32953.html#comment-238052) where I spat out my dummy and wrote “I despair at the support Lib Dems in coalition have given Gove. We all have our different last straws or red lines, but this is the reason I shall no longer be voting Lib Dem in any sort of election for the foreseeable future”.
As an aside, I believe that Labour pledged at the time to reverse the Coalition’s changes to AS and A-levels, so unless they’ve changed their minds like the Lib Dems (though I doubt they have, given that it seems Wales did not follow the same reforms that the Coalition implemented in England), perhaps there is common ground there for Layla Moran to explore.
@ David Raw,
Good to see you quoting Tony Benn! But, was he a “Little Englander” too?
@ All,
I can’t understand why any Govt would want to get themselves into this position with dodgy algorithms and esoteric statistical manipulation of results. No system can possibly be perfect in the current circumstances, but a starting point could have been a relatively simple requirement that every school’s results would have had to match, within certain limits, an average from the previous five years.
There could be an acceptance that results could err on the slightly generous side. Employers would of course be aware of this if basing employment offers on the basis of them. To counter the possibility of too many students being accepted for University courses which were unsuitable for them, universities should be encouraged to be slightly less generous than usual about allowing students to continue in the event of their first year examination results being below par.
@Laurence Cox – There are conceptual errors associated with relying on CAGs as well.
I agree, we urgently need a more robust scheme of student assessment and cross moderation of assessments between schools, to rein-in the worst excesses of teacher bias. By urgently, I mean before September 2020 so it can be used for the 2021 exam session.
What is notable in the current debate is the total lack of any media comment from the teaching unions. The only things I remember the teaching unions publicly speaking about was their concerns about social distancing and teacher holidays for schools re-opening, possibly during the traditional summer holiday. The problems with the 2020 exams was entirely predictable as soon as lockdown was announced.
@James Moore – âUniversities should have capacity for extra places as they are losing international students.â
Have you ever worked for a university?
There may be fewer international graduate students in Law and Business studies but lecturers in Law and Business Studies canât suddenly start teaching UK undergraduates in History, Physics and English.
No one (to my knowledge) are actually saying what you have inferred. Currently, universities are expecting a much reduced international student intake; leaving more places for UK resident students. The only catch, which is what the medical universities have already pointed out, is they will need additional funding for the increased number of UK resident students who don’t pay the significantly higher tuition fees international students pay.
@David Raw – I see in the Health Sector Boris Chum Dido Harding (of test and trace failure)
Her list of successive failures is significantly longer; it goes back at least to her time at Talk Talk; about her only success is repeatedly getting appointed to high profile posts…
Joe – what a good article. The LSE for example only has 30% of students from the UK. It would be fun to see how far one would get asking for a year of allocating 70% of places to students from the UK from deprived backgrounds. Answer – not very far of course!