No sooner has the largesse of REF2014 been promised than invitations to the next UK academic Olympiad are being issued. The prospect of REF2020 has unleashed a new university obsession with rankings and evaluations of research. In many departments, researchers are undergoing a process of continual audit. Every grant application is vetted; every publication is read by internal, and perhaps external, assessors. In the current system, grades are assigned 1-4, but everyone knows that the stakes have been raised and only grades 3 and 4 count. Let’s take a step back and ponder why this might not be such a good idea. Any such strategy of continuous assessment of research is bound to be inaccurate. Here are some reasons why:
- We do not know what rules are going to be applied in 2020, but they are sure to be different from 2014. Who foresaw the insertion of the ‘impact’ monkey-wrench at the last minute? Similarly, we cannot assume the current weighting given to publications in the next REF formula.
- Published work has already been assessed by the most appropriate scholars in the field during the process of peer review. In its ideal form, this is what we describe to students as ‘formative assessment’. What added value could a summative assessment from a generalist offer? It is ironic at a time when we have attempted to curb the effect of high-stakes assessments on our students, in the realisation that such exercises are of limited value in ‘the real world’, that we now impose those same stressors on ourselves.
- Any internal ranking is likely to be distorted either by relations of power within a department, or by personal animosities. Can a Reader offer a fearless appraisal of the work of a Dean? Can a colleague overlooked for promotion dispassionately review the work of someone whose success they envy?
- I think we all now know that there were some very strange GPAs and rankings emerging from REF2014. Biological sciences appear to have won the laurels, and over half the units submitted achieved a GPA of 3 or above. The sociology panel, meanwhile, delivered a harsher and more partisan verdict. Those units which reflected government priorities in their impact studies scored well; the more ‘critical’ areas of the subject were slammed. Internationally recognised departments and research groups can now legitimately be targeted for closure by their university’s management. And so a vindictive government has made vice-chancellors their willing proxies, simply by persuading them to swap evaluation of research quality for evaluation by £££££.
- Because of 4), rankings of units, and therefore of publications, is likely to follow one of two courses: scores will either reflect a privileging of government/ managerial bias against critical scholarship, or, every single subject will look at the celebrity status of biological sciences and conspire to inflate their scores in order to attain parity of esteem.
Ranking of publications is also undesirable because of the distorting effect it will have on academic work and relationships:
- It is hard enough for some scholars to submit their work to peer review and public scrutiny in the first place. If we have to contemplate the scorn of a head of department, or senior colleague – why bother? I imagine a lot of people will allow other agendas – teaching enhancement, student experience, employability to occupy them instead.
- If senior researchers – likely to be the most ‘productive’ scholars – are the ones doing all the assessing and ranking of pieces, how will they have time to pursue their own work? Inevitably, this will eat up much of their research time, and also a great deal of their ‘leisure’ time. The research capacity of many departments will begin to ebb away under the strain of constant audit.
- People will become secretive about publication plans, or leave publication until the last possible moment before the next REF. They may perseverate on the minimum 4 publications, and allow other ideas to dissipate unfulfilled and unexplored. This is unhealthy for the future of university research.
- We risk creating departments of people who despise and distrust each other. How long will Dr A hold a grudge against Professor B for a Grade 2 assessment? Will payback come when Dr A is asked to observe Professor B’s teaching?
So, it is very clear that the value of assessing work either pre- or during REF is unreliable. It is retained simply for its disciplinary effect, and because management think they can summon up research quality like a genie from a lamp. The analogy is pertinent, because they seem to imagine that a ranking score equals quality-made-visible, and that this is some pathway to quality enhancement. It will be the reverse, of course, especially when we consider the amount of research time and funding bound to be sacrificed in the process. And on top of this loss, we will create unhappy, dysfunctional departments where every colleague nurtures resentment. UK academics have seen their job satisfaction and self-esteem falling low enough. Let’s not compound the issue with another ill-founded exercise. This is hysteria masquerading as rationality. Just make it stop.
Academic Irregularities 