FE Labyrinth

Could our different inspectors please talk to each other?

Or, at the very least, could they appear to know of one another’s existence?

Halfway through the second day of the inspection, I visited a class of ten functional skills students with my designated “deep dive” inspector. They ranged in ability from entry level 1 up to Level 1. If you teach functional skills yourself, you’ll know this means it’s a challenging group to pitch a lesson to. After we left, the inspector asked me, with a knowing smile,

“Do you think that is a viable group?”

“In what sense?” I asked, thinking ahead like a chess player trying to work out where the trap was being laid.

“In terms or the range of abilities and needs in the session.”

“It’s certainly a challenge.” I replied, remaining non-committal until I knew where this round was going.

“Are you under pressure from SLT to avoid splitting the class? Is it about saving money?” The question seemed, to me at least, to imply that saving money was a grubby motive. It irked me a little and my game slipped.

“Yes, of course.” She looked surprised, as though I had willingly confessed to a crime. “We’re always under pressure to keep class sizes up. But not from SLT. When we’re audited, we’ll be challenged if they are too low.”

“Oh,” she said, “who audits you?”

What a question! Who audits us? The FE commissioner’s inspections are almost as stressful as Ofsted inspections, and the ramifications of a poor outcome are every bit as serious for the staff and students. The annual internal audits are a huge part of management’s workload at the end of the year.

This may have seemed like a small thing to the (Ofsted) inspection team we dealt with. Indeed, the off-hand comment that was quickly forgotten and we moved on to other conversations, but it highlighted to me one of the fundamental tensions running through the heart of FE. We are a public service acting like a business. Or is it the other way around? It probably depends on whether you ask a curriculum manager or the head of finance.

We struggle, under decades of funding neglect, to provide certain basics for our students. The financial pressure means we are constantly pushing for new enrolments (for new business) and pushing back at the boundaries of what we should accept in classroom standards.

The way we are inspected in FE reflects the duality of our mission. The FE Commissioner inspects the college from a perspective of financial competence. They want to know whether we are running well as a business. Their concerns about classes are limited to whether teachers are being well utilised. Are classes too small? If they are, they could be merged, and the college would make a saving. Are teachers given too much “free” time for planning and development? If so, they could be reduced in number. It’s about making sure public money is being spent efficiently.

Ofsted, however, have a different agenda. They are focussed purely on the educational experience of the students. They don’t care how much an intervention costs if it’s good for learning. An Ofsted inspector sees one-to-one sessions with struggling students and gives a big thumbs up. An auditor looks at the same thing and sees a waste of money.

As funding gets tighter, these two agendas come into increasing conflict and it would be nice if the two organisations would speak to each other. At best, they could agree on certain approaches to give a consistent message. At the least, they could make sure they both know the other one exists.

Intent, Results and the Right Inspector

This is the first of a series of short pieces exploring themes that emerged from my recent Ofsted inspection. You may want to catch up on the last inspection first (“What is the impact of that?”) for some context.

The main difference between this inspection and the last was the quality of my inspector. It was, of course, still stressful. That the stress was not added to by a feeling of being unfairly dealt with was a great relief, however. Talking to an inspector who is determined to find fault and enjoying their position of (enormous) power is grim. It reminded me that our experience of Ofsted is determined at least as much by the team implementing the framework as by the framework itself. This time, I was lucky.

Whereas last year the defining experience (for me) was a surreal conversation about “impact” with an inspector who did not appear to fully understand the statistics she was grilling us on, this time the conversations revolved around teaching methods and evidence of whether students were learning or not. Where my inspector was critical, I felt (in most cases) that the criticism was fair. That is certainly an improvement, though there were still contradictions and frustrations during the visit.

One thing was clear from the beginning. They pointedly did not wish to see internal data. My own ‘deep diver’ at least gave me the opportunity to present her with some data, but I got the impression it did not feature heavily in her decision making. Another colleague’s deep diver put it more bluntly.

“We’re not looking at data,” he said to my fellow manager, who was trying to present him with a ring binder full of impressive results.

Having read about the new framework, I was not surprised to find that data had dropped in importance. However, I expected them to make some concession to the very advice they gave us when we were last inspected, only one year ago. They told us to improve certain results relative to the national average, improve internal data gathering and make it easier to see “at the click of a button,” how individuals and courses were performing. We worked hard on realising those goals. We presented Ofsted with the result of a year’s work to those ends, and they weren’t interested. We were even told that we should not focus too much on “just getting the results.” Well. Schools and colleges up and down the country did not start obsessing over national averages for no reason. In the long run, I welcome the reduced focus on data. I could not help wondering, though, what Ofsted would make of it if I told a class to go away and revise algebra for a week, and then tested them on geometry when they returned. To complete the analogy, I could feign shock at my students’ inevitable outrage and respond with a gasp,

“But you weren’t only revising algebra because I told you to, were you? I mean, you really should have been revising geometry too, because you love the subject so much.”

What were they looking at, then, if they weren’t interested in data? Curriculum planning. They were keen to understand why we put the curriculum together in the order we chose. Fortunately, we have given that a lot of thought. Our students come to us having already sat through maths and English several times, so it’s sensible for us to think differently about sequencing. I did wonder what this conversation is like for my counterparts in secondary schools, though. Does a head of maths in Yorkshire need to have clear reasons for their sequencing which are different, say, from a head of maths in Sussex? Is there something in the air in different regions that means mathematics needs to be approached in a different way? Is it so bad to download the exam board’s recommended scheme of work and go with that? I’m not convinced it is.

I tried to imagine how the ‘curriculum intent’ conversation might go for a head of maths who said,

“For four thousand years, people have needed to know about finding factors before they could cancel fractions, so we do factors before we do fractions.”

It seems odd to me that Ofsted want every school to reinvent the wheel and come up with a unique curriculum sequence, when we are all delivering the same knowledge. If there is a specific order in which it’s best to do things... could they just tell us what it is? Perhaps this conversation makes more sense in subjects like history and geography, which might indeed be thoughtfully modified to account for location.

Their other interest was what students could remember. They pulled them out of lessons and asked them maths questions, sometimes about topics they studied three months ago. I quite liked that. What better way to test if they are learning? Where the inspector deemed learning to be poor, it usually matched my judgement. My only quibble is that it places huge stakes on a small number of conversations. Where such importance is placed on anecdotal evidence it is bound to lead, sometimes, to inaccurate judgements. It is also easy to steer these “conversations” in such a way they provide confirmatory evidence of a judgement the inspector has already made. This brings me back to the quality of the inspector. In my case, she was open to changing her mind and being directed to additional evidence. She seemed admirably determined to make a fair, objective judgement. But what if the same inspector as last time had been conducting those conversations? It doesn’t bear thinking about.

Finally, there was one contradiction running through the whole inspection. Do results matter now or not? The inspectors made it clear they were not interested in data, either internal or external. They also insisted that they did not want to see “exam hot-housing,” and “teaching to the test.” However, when we asked why they had chosen the courses they did for a “deep dive” it was because attainment figures had recently slipped back. So, if your exam results are not good enough, they will come in to check that you are not focussing too much on exam results. There is a clear contradiction here and it highlights my feeling that Ofsted have not yet decided for themselves whether and how much results matter.

The Infinite Resource Machine

Last week I was looking for a worksheet on the general term of a linear sequence. A quick search on the usual website turned up some interesting results. Among the various options, there were three resource bundles ranging from £1 to £3 that looked suspiciously similar. Not wanting to buy them merely out of curiosity, I clicked on the preview pages. Two were identical. So, one person at least was profiting from stolen work. This is quite common on resource sharing (and selling) websites. Why? There’s a bit of background to fill you in on first about why these websites are so heavily used. When I attended the first lecture of my teacher training course, the advertised topic was “differentiation.” It amuses me to recall that, since I was training to be a maths teacher, I thought this might be a session on how to teach basic calculus. “Differentiation,” (as a pedagogical practice) was still a new thing in those days. It wasn’t one of those fads that comes and goes in the space of five years, though. Differentiation has stuck around. The theory behind it all sounded perfectly plausible when the lecturer revealed it to us: every student is at a different stage of development in their understanding of a given topic, and the exercises you give them need to reflect their particular needs. Of course, I had not yet taught a lesson, so I was still thinking very much in the abstract. The practical ramifications of this new philosophy only occurred to me when I began observing lessons in school.

During the first lesson I observed there were at least five different worksheets being completed at any one time (the teacher may have been trying to impress). Gone were the rugged textbooks with their bent spines and phallic illustrations from which I had worked throughout my own time in school. Everything was printed on sheets, with colour-coding at the top to identify the right level of challenge. I was struck by how much paper was being used in every lesson. “Where do you get these sheets from?” I asked. “Oh, I like to make them myself, so I know they’re appropriate for the class I’m teaching,” came the suave and terrifying reply. (He was definitely trying to impress).

When I prepared to teach my own lessons, the expectation appeared to be that I would craft my own resources, tailor made to the precise educational requirements of the students I knew. Not only did I have to provide at least three different levels of difficulty, but it had to be appropriate “for that particular class.” In other words, if I didn’t want to justify exactly why I had chosen a specific worksheet, it had to be new. It was both disheartening and impossibly time-consuming. Preparing for a single lesson took hours and the end result was often terrible in comparison to the professional quality of a publishing company’s book. But when I relayed these difficulties to my peers on the PGCE program, there followed a great deal of hilarity. “You’re actually making your own worksheets?!” A friend asked me, all but wiping tears from his eyes, “Why don’t you just use TES?” That was when I discovered what had replaced the humble textbook. It wasn’t teachers up and down the country busily writing their own worksheets for their unique students who were confused in such unique and original ways that an entirely new set of practise questions was called for. Nor was it the “online textbooks,” that have been rolled out with zero impact year after year since the digital revolution began. It was the vast repository of original, copied, edited and reprinted resources that you can find on the Times Education Supplement website. The discovery of this cache of ready-made worksheets was a great relief to me. I have since found other reliable websites as well. Soon I was able to find what I needed for a lesson, put my name on it and print thirty copies, in under ten minutes.

The problem with this approach is that I wasn’t really tailoring my questions to the specific class in front of me. I was merely pretending to (which turned out to be just fine by my observer, actually). In fact, quite often, I was planning my lessons around what resources I could find at the last minute, in such a way that said resources would appear to have been designed for the class and lesson. It’s interesting to look at which worksheets get the most downloads. The ones with dense rows of questions, resembling a page from an old-fashioned textbook, seem particularly popular. Add an answer sheet (like the back page of a textbook has) and you’ve got a hit, top-rated resource. It’s true that Ofsted have changed their tune about textbooks in recent years, but the culture of education is too saturated now with suspicion of those basic, wonderful tools, for them to make a widespread comeback as the staple classroom resource. I’ve had colleagues who photocopied pages from books when they wanted to use them, so they could hand out sheets rather than risk appearing as the kind of teacher who “does textbook lessons.”

It is this environment in which the TES resources website has become so heavily used and, more recently, monetised. Lessons sold at a premium of up to £5 at the top end are often sold with the promise that they won an “outstanding” grade in an observation. Worksheets are always “Differentiated.” Often, though, these resources are also copied. The editability of the worksheets is part of the appeal, but documents sold in easily edited formats like Word or Powerpoint can also be easily re-sold under a different name. Original creators of content have found there is very little they can do when they see someone else selling their work.

All this individual purchasing from teacher’s own pockets, copying, editing and endless photocopying is not done to improve the education of our students. It is done to circumvent the weird taboo that has emerged in teaching around relying on a textbook. Somewhere, in a parallel universe in which that taboo never arose, teachers are wasting a whole lot less time, less paper is being shunted from the copier to the bin, less money is being spent – and students are having almost exactly the same experience.

How do they get those results?    The national pass rate for GCSE maths re-sits in FE is currently around 16%.  Less than one in every five students having a second (or third, or fourth) go at achieving a grade 4 actually gets it.  For some time, I’ve been looking for a shining example of a college which bucks this trend. 

Last year I saw an advert for a position at a nearby rival (we’re corporations, remember – that's how we think of each other).  The advert highlighted that the successful applicant would be joining an “outstanding” maths team.  To be honest, I don’t want to work in an “outstanding” department.  I’ve done so before and hated it.  I’d rather work in a really good one, but that’s a difference to explore another time.  Though I did not apply for the job, it led me to wonder what their results were like.  I looked them up.  They were much higher than the shockingly low national rate.  How, I asked myself, have they done this?  And why are they not famous for it?  Why are they not travelling the length and breadth of the country showing the rest of us where we err? 

I did a little ferreting around and found my answer.  I am sure their staff are dedicated and capable (outstanding, even) but that is not what explained the high results.  It was when I found their enrolment strategy, passed to me by a friend whose partner worked there, that I said “Ah! That’s how they do it!”  This brings me to the first of three strategies I have come across for improving results without making anyone better at maths. 

They enrol everyone for GCSE maths, including the students who already have a grade 4.  The vast majority of those with a grade 4 or above go on to achieve exactly the same grade they already had – but it still counts in that college’s attainment figures!  This sounds like a trick too easy to get away with, but Ofsted were full of praise for the policy.  “All students are given the opportunity to improve their maths,” the report stated.  In a way, that was true.  Except teaching hours for those students were set at the bare minimum and a blind eye was turned to non-attendance figures which were (I am told on the grapevine) somewhat massaged at the end of the year.  It’s not surprising they didn’t make any progress.  They didn’t need to.  All the college required was for them to tread water and it would make them look wonderful.  It’s a cynical policy, chosen for cynical reasons – but easy to present in a positive light to an inspector on a tight schedule who likes the statistics it produces. 

Here’s another method some colleges employ to improve their results, this time in retrospect!  Suppose you have 100 students sitting GCSE maths and only 16 of them pass.  You would think that’s a pass rate of 16%, right?  Not necessarily.  A truly outstanding maths team does not give up just because the exam was two months ago.  What if 30 of the students who failed return to study with you the following year – in this case, you up can retrospectively decide that those students were on a two-year course.  That means they’ll count in next year’s attainment figures and now, all of a sudden, your 16 passes are out of 70 instead of 100.  Your pass rate just went up to 23%.  Congratulations! 

These students are called “rollovers,” and they are an administrative pain in the rear end, as they have to be enrolled on year-old codes and tracked and monitored separately even though they are in the same classes as everyone else.  It’s more grist for the mill of the vast FE bureaucracy, more man-hours and salaries spent on form filling and paper chasing that benefits none of the people using the service but is so endemic within it. 

Finally, one of the main measures of success for colleges is the general attainment rate.  This is simply the percentage of all students who pass their qualification.  It makes sense for BTECs and other vocational courses to be judged in such a way (apart from the powerful incentive that gives to fudge the in-house assessments and coursework), but it makes no sense at all that GCSEs are rolled up into that figure.  A student currently at grade 2 in GCSE could do either Level 2 Functional Skills (L2 FS) or GCSE under the new conditions of funding.  However, no head of faculty I know wants their students in a L2 FS class.  The reason?  A grade 2 student might not get a grade 4 in their exam this year, but even a grade 1 counts as a positive achievement for the department.  If you fail L2 FS, it doesn’t matter how close you were, you go down as a negative.  So, even when a student has failed GCSE three times in a row but might pass L2 FS, people often won’t risk putting them in for it.  Nobody trusts an inspector to look past the headline figure and try to work out what’s going on behind the scenes.  Having experienced a number of poor inspections, I can’t say I blame them. 

All of the above tactics are, as far as I am aware, legal if not entirely above board.  There are lots of decisions made in FE that fall into a sort of grey area, where you’ll get away with them if the college generally smells good and is not in financial trouble.   

What also boggles my mind is how easy these practises would be to stop.  You could end all three with two simple steps: 

1) Stop counting GCSEs as an achievement if the student doesn’t improve their grade.   

2) Insist that all students who sat an exam count on the figures, with no exception for “rollovers.”   

I cannot fathom why these loopholes persist and are so easily exploited.  The only conclusion I can draw is that the government has simply taken its eye off FE and doesn’t care that much what happens here.  Colleges are all businesses now, so as long as they are financially stable that’s all that really matters.   

How We Make the Predicted Grade Sausage – A walkthrough for those wondering whether to ask their teams to produce one   Now the exams are over, all that is left this year is to prepare for the next.  It’s a chance to clear out cluttered rooms, re-write schemes of work and print resources for the first weeks back – and of course, because this is the world of education, to gaze into our crystal balls and predict what will happen on results day. 

Most governors and SLT like to know in advance what result the students will get.  Some are content with broad achievement predictions.  Others are keen to know how many will achieve a grade 4, how many a grade 5 and so on.   

I’ve been in colleges where the teachers have refused to guestimate grades, and bitter feuds have arisen between the unions and SLT.  I’ve been in colleges in which the accuracy of a teacher’s predictions were used to analyse how well they “knew their students.”  I even heard of a secondary school which made its year 11 students complete the GCSE paper they had just done the following week, in order to mark them and know in advance how well the cohort had performed!  Apparently, the students downed tools and walked out.  These are the lengths we go to in order to see two months into the future.  Why?  Perhaps because at this stage there is nothing more we can do, and nobody likes to feel as though they are doing nothing. 

The following paragraph is a confessional.  It’s a day in the life of the data sausage factory.  I hope to convince you that it’s not worth predicting in advance how well your school or college will do.  You can’t know.  All you can do is create work and acrimony with staff.  Or, at least, if you are going to do it, think about why staff might be afraid to present you with the truth and how might that affect the numbers?  Think about whether you want a realistic picture, or something to reassure you before the summer break. 

It usually works like this:  SLT asks upper middle management (UMM) for the performance predictions.  UMM make a beautiful spreadsheet, full of top-notch conditional formatting and delightfully complex sub categories of grading and ask lower middle management (LMM) to get it filled in. LMM take it to the lecturers who say they have no idea what the students will achieve.  LMM say we can’t tell that to UMM, just put your best guess in.  Some lecturers spend an hour on it, thinking over and over about which student will get what grade.  Some spend minutes on it (and are possibly more accurate as a result).  Others predict that everyone will fail on the grounds they will be held to account if they predict passes which they don't deliver.  Some lecturers predict that everyone will pass, because it’s just easier and they are retiring next week and by the time those chickens come home to roost it will be someone else's nest.  LMM takes this hot mess back to UMM, who declares that it simply won’t do.  You can’t predict a result that’s too low (or too high) or the next few weeks will be hell.  LMM reads between the lines and asks their teachers for another prediction with some tacit guidance on the figure they should ideally reach.  Some lecturers comply, but others refuse to fill it in a second time.  LMM changes a few 3s to 4s and 4s to 5s because it’s getting late and they want to go home and it doesn’t matter anyway.  They come across a handful of students that nobody claims to have taught and randomly select grades on the basis of how familiar their name is, or which song is on spotify at the moment, or what they gave the last three students and how that will affect the mean.  LMM sends the sheet back to UMM, who does some further ‘tidying’ to make sure the bottom line is more or less 0.5% better than last year, and submits the sheet back to SLT. 

SLT know, on some level, that the data is absolute crystal bollocks.  They know this because they were once lecturers or managers doing what it took to keep the next level up off their backs.  The Governers, perhaps, are the only ones who take it seriously and are concerned on results day when at least one of the measures is significantly different from what was predicted and nobody else is in the least bit surprised. 

I’m sure there are schools and colleges around today that don’t bother with this dance anymore, but we still do it every year.  It’s one of the curses of technology.  Just because a tool exists, doesn’t mean it has to be used.  Just because you could make a really nice spreadsheet for gathering predicted grades, does that mean you should?  It is a ritual with built in layers of decision making that exist to protect SLT from seeing something they won’t like.  It would be better to just wait until results day.