How We Make the Predicted Grade Sausage – A walkthrough for those wondering whether to ask their teams to produce one Now the exams are over, all that is left this year is to prepare for the next. It’s a chance to clear out cluttered rooms, re-write schemes of work and print resources for the first weeks back – and of course, because this is the world of education, to gaze into our crystal balls and predict what will happen on results day.
Most governors and SLT like to know in advance what result the students will get. Some are content with broad achievement predictions. Others are keen to know how many will achieve a grade 4, how many a grade 5 and so on.
I’ve been in colleges where the teachers have refused to guestimate grades, and bitter feuds have arisen between the unions and SLT. I’ve been in colleges in which the accuracy of a teacher’s predictions were used to analyse how well they “knew their students.” I even heard of a secondary school which made its year 11 students complete the GCSE paper they had just done the following week, in order to mark them and know in advance how well the cohort had performed! Apparently, the students downed tools and walked out. These are the lengths we go to in order to see two months into the future. Why? Perhaps because at this stage there is nothing more we can do, and nobody likes to feel as though they are doing nothing.
The following paragraph is a confessional. It’s a day in the life of the data sausage factory. I hope to convince you that it’s not worth predicting in advance how well your school or college will do. You can’t know. All you can do is create work and acrimony with staff. Or, at least, if you are going to do it, think about why staff might be afraid to present you with the truth and how might that affect the numbers? Think about whether you want a realistic picture, or something to reassure you before the summer break.
It usually works like this: SLT asks upper middle management (UMM) for the performance predictions. UMM make a beautiful spreadsheet, full of top-notch conditional formatting and delightfully complex sub categories of grading and ask lower middle management (LMM) to get it filled in. LMM take it to the lecturers who say they have no idea what the students will achieve. LMM say we can’t tell that to UMM, just put your best guess in. Some lecturers spend an hour on it, thinking over and over about which student will get what grade. Some spend minutes on it (and are possibly more accurate as a result). Others predict that everyone will fail on the grounds they will be held to account if they predict passes which they don't deliver. Some lecturers predict that everyone will pass, because it’s just easier and they are retiring next week and by the time those chickens come home to roost it will be someone else's nest. LMM takes this hot mess back to UMM, who declares that it simply won’t do. You can’t predict a result that’s too low (or too high) or the next few weeks will be hell. LMM reads between the lines and asks their teachers for another prediction with some tacit guidance on the figure they should ideally reach. Some lecturers comply, but others refuse to fill it in a second time. LMM changes a few 3s to 4s and 4s to 5s because it’s getting late and they want to go home and it doesn’t matter anyway. They come across a handful of students that nobody claims to have taught and randomly select grades on the basis of how familiar their name is, or which song is on spotify at the moment, or what they gave the last three students and how that will affect the mean. LMM sends the sheet back to UMM, who does some further ‘tidying’ to make sure the bottom line is more or less 0.5% better than last year, and submits the sheet back to SLT.
SLT know, on some level, that the data is absolute crystal bollocks. They know this because they were once lecturers or managers doing what it took to keep the next level up off their backs. The Governers, perhaps, are the only ones who take it seriously and are concerned on results day when at least one of the measures is significantly different from what was predicted and nobody else is in the least bit surprised.
I’m sure there are schools and colleges around today that don’t bother with this dance anymore, but we still do it every year. It’s one of the curses of technology. Just because a tool exists, doesn’t mean it has to be used. Just because you could make a really nice spreadsheet for gathering predicted grades, does that mean you should? It is a ritual with built in layers of decision making that exist to protect SLT from seeing something they won’t like. It would be better to just wait until results day.