The Disrespect of the TEF

 

I have been a few days late to the sector-wide freakshow that is the TEF results.  There has been a news-grabbing but probably temporary perturbation to the traditional hierarchy of mostly English universities, their VCs enlivened by the prospect of being able to raise tuition fees. The resulting categorisations of Gold, Silver and Bronze brought forth just one expression of outrage from Sir Christopher Snowden, the VC of the University of Southampton, and former Universities UK president. Altogether three more (Liverpool, Durham and York) have joined him in launching an appeal. Endorsed by the vice-president of the NUS, Sorana Vieru, Snowden levelled this criticism, “I know I am not alone in having deep concerns about its subjective assessment, its lack of transparency, and with different benchmarks for each institution removing any sense of equity and equality of assessment.”

The statement would have had more force if it had been delivered before the data for the TEF had been sent to Hefce by compliant vice-chancellors. Better still, the sector could have prevented needless and undeserved reputational damage to a majority of universities which received Bronze and Silver ratings if they had stuck together and paid heed to the uncontestable arguments against the TEF. The Council for the Defence of British Universities (CDBU) has published several informative pieces on their website. A key contributor is Professor Dorothy Bishop of the University of Oxford whose argument is summarised in this excellent reply to an article by Edward Peck, VC of Nottingham Trent University.

Her key points are that the justification for the TEF of supposed ‘lamentable teaching’ is unfounded, and in any case, weaknesses in teaching can be diagnosed and addressed by the current QAA inspection and quality framework. Crucially, “The validity of the National Student Survey as a measure of teaching quality has been roundly criticised, and these criticisms appear to have been accepted by the chair of the TEF.” Also, “the statistical properties of NSS data have been described as unsuitable as a measure of teaching quality by the Office for National Statistics, the Royal Statistical Society and, most recently, by Lord Lipsey, chair of the All Party Parliamentary Group on Statistics.”

This should have been a damning enough condemnation to stop the TEF in its tracks before the HEAR Bill could be hustled through parliament before the General Election. It is disturbing that a process which would not get through the first pass of peer review if submitted to an academic journal is now being triumphantly lauded as begetter of a new and ‘disruptive’ hierarchy of universities which better reflects and serves ‘student choice’.

The majority of those who teach in universities are clear that the TEF methodology is fatally flawed, and the results meaningless. There are those in government and the media, though, who still defend its meager claims on credibility. One such is Nick Hillman, Director of HEPI, the Higher Education Policy Institute, a position he assumed after a spell as the Special Advisor to David Willetts when the latter was Minister of State for Higher Education. Nick appears, from my many years of interaction with him on Twitter, to be an amiable and fair minded human being. I appreciate his willingness to engage with critics of the TEF and higher education policy generally. However, I was alarmed at some comments made during this thread. It is worth showing the interaction in full below. The thread starts with the CDBU taking issue with Nick’s view that the TEF offers students important information on which to base their choice of university, a view which echoes the wording of the HE White Paper of May 2016.

What surprised me was Nick’s response to my suggestion that an ill-conceived array of proxy metrics might not serve anybody’s purpose. He seems to believe that it is too much to ask that any policy should work well from Day 1. I suggest that a well-recognised business process, Six Sigma, which focusses on identifying flaws and process improvement, could be adopted here. In an era when public sector organisations are directed to be more business-like in terms of efficiency and effectiveness, I would have thought this would be an article of faith in policy analysis too. Clearly, getting it right in government circles is too much to ask, as Nick reflects on “the messiness of policymaking in a democracy”. Actually, you don’t need anything as systematic as Six Sigma, you just need evidence-based policymaking, and it starts with paying attention to well-informed critics like Dorothy Bishop.

My point in writing this is not to make a personal attack on Nick, who, as I said above, distinguishes himself by his willingness to engage with alternative viewpoints. This is rather a howl of despair at an apparent double standard in expectations between government and those who must deliver their policies. What I am hoping to convey is the frustration of academics who are asked to restrain their own exploratory instincts in favour of a highly-regulated, audited and disciplined approach to working which has predictability and safety as its guiding principles.

What an ironic reversal this is. In previous years we might have expected universities to have license to experiment, to fail, to be messy in their approach. This was how discovery and progress was assumed to move forward. No longer, apparently. While government is afforded the indulgence of failure in policy-making, this is not the case for those who are charged with delivery. For policy makers there appears to be no obligation to embody ‘excellence’, to do what works, to get it right first time, to be evidence- and outcomes-led. These are instead demanded from those who must fulfil them under conditions of an increasingly demanding workload into which more government policy initiatives are emptied on a continuous basis. Unlike government, there is no five-year window for memory slippage when you are subject to performance improvement procedures every three months over some imagined lapse in ‘excellence’. It must be an enjoyable life, driving policy changes when you know that ‘messy’ outcomes will be tolerated, but it is a rather different story in academia 2017.

Let me give just a couple of examples. Research can be a step into unknown territory, but in REF culture, it would be an unwise scholar who set out without a clear sense of – not just results – but the ‘value’ and ‘impact’ of their work. No unpredictable outcomes possible. And similarly, academic staff are expected to fund their research with grant money from research councils with success rates for applications as low as 11%. Similarly, in our teaching, despite recognising that each individual student will take something different from our courses, we must submit them to a system of empty standardized ‘learning outcomes’. And when we have assessed them in a way which belies their imagination and intellectual response, we must endeavor to portray this as ‘personalisation of the learning experience’. There is little tolerance for even necessary ‘messiness’ in academic life. We mourn its passing.

Nick has urged universities to offer more support for students with mental health difficulties in a recent HEPI report. I look forward to another HEPI report which considers the crisis in academic staff mental health, and the role of frameworks such as the TEF and REF in heightening this. They may seem like benign instruments of audit, necessary to justify the considerable public spend in higher education. This would be uncontroversial but for the fact that these have been folded into the disciplinary mechanisms of New Public Management in universities. They too have been personalized and dashboarded into instruments of performance management.  And if HEPI does join the growing band of voices advocating for a more humane university workplace, I hope the report encounters a more gracious reception than I did when I spoke truth to power.

Nick seems to find my retort disrespectful, and if I have been, I offer my apologies. I have always said that the only thing that ever trickles down is contempt, and academics feel it raining down from government, magnified by sections of university management.  As I indicted in my response, democracy has perished in universities alongside ‘messiness’. And when such a double standard is in place, and you are only as good as your last ‘win’, even in the face of structural obstacles, it is nothing less than abusive.

Advertisement