The conceptual error behind the A level fiasco

On day zero we heard of some horrendous downgrades. Perhaps that was just anecdote. No longer. Clearly now we are in the realm of overwhelming evidence that the awarded grades are not a fair indication. Every single FE college responding to a survey covering half of all colleges report lower grades than in previous years. Large numbers of independent schools are bragging about record high grades. The stolen A*s have been awarded to others and the top university places have therefore largely been taken.

In principle, an algorithm could be fair from one school to the next, giving effectively each school its prior attainment. Clearly the algorithm hasn’t done this and therefore clearly it contains errors.

Even without these errors, the concept that if your centre expects a U to be awarded, then somebody has to be given a U, however well they are actually performing, is grotesque. But this is the concept, and it applies to all the grades below the level you are performing at, not just the Us.

The idea that the formula doesn’t work for small cohorts is correct. The remedy of granting inflated grades to students in small cohorts (largely public schools) and deflated grades to everybody else, is worse than the disease. The formula does not work for large cohorts either.

This brings me to the final conceptual error in the whole enterprise. Had the formula been correct, and had a fair solution been found to the problem of small cohorts, we would have been in a position where each school got broadly the results they would normally get, and we might all shrug and say ‘well that seemed to work’. Universities would have right number of applicants meeting their offers and would be happy.

But even then there would be no fairness from one individual to the next. The hard working late bloomer would be beaten by the coaster every time. There would be disappointed students, but we expect that anyway. There would be no proof a student could offer that they had mastered their subject. We would have fixed the aggregate statistics to look fair, and betrayed the individual students. The unfairness this fiasco has uncovered is only the tip of the unfairness iceberg.

How was this ever thought to be acceptable? I think the malign influence of collectivism has caused us to think of fairness in aggregate terms rather than individual terms: if somebody else from your school gets your grade, that doesn’t matter because they are the same as you, as far as the stats are concerned.

But the only fairness that matters is fairness to individuals.

Now that uni places have already been offered, I think the only remedy is to revert to CAGs (school assessed grades) – as Kirsty Williams has announced will happen in Wales – and to fund an expansion in places. Universities that have taken the invented grades at face value may deserve a bloody nose but it is more important to fund the extra places anyway to remedy the injustice to individual students. Universities should have capacity for extra places as they are losing international students.

* Joe Otten was the candidate for Sheffield Heeley in June 2017 and Doncaster North in December 2019 and is a councillor in Sheffield.

Read more by .
This entry was posted in Op-eds.
Advert

25 Comments

  • For years we have been using the exam system as a means of and a pretext for denying young people the chance to follow their dreams. There will never be a better time than now to tear up the whole thing and start again.

  • Peter Watson 17th Aug '20 - 4:43pm

    “But the only fairness that matters is fairness to individuals.”
    I love this line!

  • Laurence Cox 17th Aug '20 - 4:50pm

    There are conceptual errors associated with relying on CAGs as well. While many, perhaps most, teachers have given their students accurate grades; some have obviously been wearing rose-tinted spectacles otherwise we would not have seen the grade inflation that led to the downgrades in the first place. Some students will not go on to university and for them, those predicted results will become the data that their potential employers rely on to choose between similarly-qualified applicants. Do they deserve a bloody nose as well? At least the students that go on to university will have another opportunity in three years’ time to demonstrate their qualities.

  • Peter Watson 17th Aug '20 - 4:54pm

    @Chris Cory “For years we have been using the exam system … There will never be a better time than now to tear up the whole thing and start again.”
    One of the things that upset me most about the Lib Dems in Coalition is that they seemed to align themselves ( once I would have said we aligned ourselves, sigh 🙁 ) too closely with Michael Gove’s reforms and an increased emphasis on terminal exams at 16 & 18. It makes some of the criticisms this week ring hollow, but as you point out, there is probably an opportunity now to push for radical change if only someone can be clear about what that change should be.
    But we’ll probably just get a review and then a recommendation that teachers’ administrative burden is increased by carrying out the rigorous prediction exercise every year alongside the exams.

  • Excellent post, Joe. And I am so pleased that Ofqual has done a U-turn, although, as you imply, that still leaves a problem for universities that may have to honour more offers than they were expecting. Under the current rules they actually get fined if they take more than their quota.

  • @Laurence Cox – yes, indeed, there are some issues with teacher assessments including some deliberate overestimates in some schools. But remember that this year there was nothing else to base the final grade on. But teacher assessments are overall a much fairer way of doing things than any of the possible alternatives.

  • Peter Watson 17th Aug '20 - 6:08pm

    @Mary Reid “But teacher assessments are overall a much fairer way of doing things than any of the possible alternatives.”
    That’s the crux of it. I don’t think there is any approach that is absolutely fair to all interested parties, including past and future pupils.
    I’ve little sympathy for past, present and future Tory Education Secretaries, but I’m also unimpressed by the demonisation of Gavin Williamson in some quarters. Like Kirsty Williams, John Swinney, and Peter Weir, he’s had to deal with an unprecedented situation (of which year 11 & 13 school exams are only a part, albeit an important one) and at the same time they and the employees of every organisation involved (like many of us!) have had to cope with novel working practices. I really don’t envy any of them!

  • John Marriott 17th Aug '20 - 6:12pm

    Now that they are going to use teacher assessments, is it a case of the triumph of man over the machine?

  • James Moore 17th Aug '20 - 9:05pm

    “Universities should have capacity for extra places as they are losing international students.”

    Have you ever worked for a university?

    There may be fewer international graduate students in Law and Business studies but lecturers in Law and Business Studies can’t suddenly start teaching UK undergraduates in History, Physics and English.

    Lecture halls and labs can’t suddenly be built to accommodate larger cohorts in some subjects, nor can staff be readily found to teach them. That’s before you consider the problems of overcrowded library and computer facilities in a Covid world. Courses start in less than a month remember…

    Let’s have a grown up discussion rather than just lumbering universities with the problem.

  • I believe it is the PM’s chief advisor who believes in the reliance data-driven decision making – government by algorithm is right up his street. Now the government has been forced to do what it has been elected to do (and what all democratic governments are elected to do), and use its discretion. Never mind Covid 19, there is a potentially dangerous outbreak of common sense out there.

  • Catherine Smart 18th Aug '20 - 8:33am

    “But the only fairness that matters is fairness to individuals.” ABSOLUTELY.
    The fundamental error was made right at the start – when Ofqual assumed that its task was the same as in ordinary years – to standardise between years: and the Government did not tell them otherwise. But this year is not an ordinary year – and never can be. It should have been made clear right from the start that the task was to be as fair as possible to the young people unfortunate to be coming to this stage of life in this most extraordinary year.

  • George Thomas 18th Aug '20 - 9:50am

    In Wales, 94% of students would have been covered by one of: i) their teacher’s predicted grade, ii) a higher grade as calculated through the algorithm, or iii) their assessed AS work. The 6% badly treated by this system, and those disagreeing with being marked on their AS work, could appeal for free to ensure fairest possible result….until England and Scotland decided to use teacher predicted grades only which creates unfairness through comparison. The majority of the problem in Wales rests on it being seemingly a last minute decision (surely we could have had results and time to appeal before Universities started to fill up places?) and that AS grades weren’t immediately sent to Universities so places were still lost, however this should not be a case where analysis says all major parties failed as much as each other: Wales’ system was most robust, Scotland handled the PR best and England made the worst decisions and in the worst style.

    The problem has shifted now so that Universities and BTEC students are left in a confused and disadvantaged state. This hasn’t finished yet despite, some, politicians attempts to blame the computers and move on.

  • Joe hits the nail on the head; governments have been building a system increasingly focussed on using tests and statistics to measure school performance, instead of using them to help the learning and progress of people. The move to over reliance on end exams is a fundamental error also and Sam Freedman (former adviser to Michael Gove) admitted that on newsnight last night. We have here the core of a good message to the public in the relevant local elections next May about our local school systems.

  • One of the grosser outcome of this fiasco was post code lottery discrimination.

    I am reminded of Tony Benn’s old nostrum : “Those in positions of economic, social and political power should always be asked five questions:

    “What power have you got?” “Where did you get it from?” “In whose interests do you use it?” “To whom are you accountable?” “How do we get rid of you?”

    Those questions should be asked of the Academy Chains…….. It’s time to return schools to democratically accountable L.E.A.’s., and what is the stance of the Lib Dem Leadership hopefuls on this ?

    I see in the Health Sector Boris Chum Dido Harding (of test and trace failure) has been given a nice little earner. The Benn questions should be asked there too.

    Dido Harding, a Conservative peer who heads up England’s widely criticised test-and-trace system, has been chosen to run a new institute to replace Public Health England, after the controversial decision to axe the agency.

    Harding will be named as the chair of the National Institute for Health Protection, which will be charged with preventing future outbreaks of infectious diseases, despite the poor performance of NHS test and trace, which she has led since May.

  • Peter Watson 18th Aug '20 - 1:57pm

    @Steve Comer “Yet before the new system had time to properly bed in, along comes Gove and Cummings who rip it up and revert to the old system”
    I completely agree with the sentiment, but unfortunately Gove & Cummings were ably assisted by a Lib Dem Schools Minister, David Laws.
    I recently came across an old post of mine on this in Feb 2013 (https://www.libdemvoice.org/opinion-goves-a-level-reforms-risk-pushing-many-universities-out-of-reach-32953.html#comment-238052) where I spat out my dummy and wrote “I despair at the support Lib Dems in coalition have given Gove. We all have our different last straws or red lines, but this is the reason I shall no longer be voting Lib Dem in any sort of election for the foreseeable future”.
    As an aside, I believe that Labour pledged at the time to reverse the Coalition’s changes to AS and A-levels, so unless they’ve changed their minds like the Lib Dems (though I doubt they have, given that it seems Wales did not follow the same reforms that the Coalition implemented in England), perhaps there is common ground there for Layla Moran to explore.

  • Peter Martin 19th Aug '20 - 10:50am

    @ David Raw,

    Good to see you quoting Tony Benn! But, was he a “Little Englander” too?

    @ All,

    I can’t understand why any Govt would want to get themselves into this position with dodgy algorithms and esoteric statistical manipulation of results. No system can possibly be perfect in the current circumstances, but a starting point could have been a relatively simple requirement that every school’s results would have had to match, within certain limits, an average from the previous five years.

    There could be an acceptance that results could err on the slightly generous side. Employers would of course be aware of this if basing employment offers on the basis of them. To counter the possibility of too many students being accepted for University courses which were unsuitable for them, universities should be encouraged to be slightly less generous than usual about allowing students to continue in the event of their first year examination results being below par.

  • @Laurence Cox – There are conceptual errors associated with relying on CAGs as well.
    I agree, we urgently need a more robust scheme of student assessment and cross moderation of assessments between schools, to rein-in the worst excesses of teacher bias. By urgently, I mean before September 2020 so it can be used for the 2021 exam session.

    What is notable in the current debate is the total lack of any media comment from the teaching unions. The only things I remember the teaching unions publicly speaking about was their concerns about social distancing and teacher holidays for schools re-opening, possibly during the traditional summer holiday. The problems with the 2020 exams was entirely predictable as soon as lockdown was announced.

    @James Moore – “Universities should have capacity for extra places as they are losing international students.”
    Have you ever worked for a university?
    There may be fewer international graduate students in Law and Business studies but lecturers in Law and Business Studies can’t suddenly start teaching UK undergraduates in History, Physics and English.

    No one (to my knowledge) are actually saying what you have inferred. Currently, universities are expecting a much reduced international student intake; leaving more places for UK resident students. The only catch, which is what the medical universities have already pointed out, is they will need additional funding for the increased number of UK resident students who don’t pay the significantly higher tuition fees international students pay.

  • @David Raw – I see in the Health Sector Boris Chum Dido Harding (of test and trace failure)
    Her list of successive failures is significantly longer; it goes back at least to her time at Talk Talk; about her only success is repeatedly getting appointed to high profile posts…

Post a Comment

Lib Dem Voice welcomes comments from everyone but we ask you to be polite, to be on topic and to be who you say you are. You can read our comments policy in full here. Please respect it and all readers of the site.

If you are a member of the party, you can have the Lib Dem Logo appear next to your comments to show this. You must be registered for our forum and can then login on this public site with the same username and password.

To have your photo next to your comment please signup your email address with Gravatar.

Your email is never published. Required fields are marked *

*
*
Please complete the name of this site, Liberal Democrat ...?

Advert



Recent Comments

    No recent comment found.