Raising the Bar: The Metric Tide That Sinks All Boats

Liz Morrish writes: A longer post than usual, but very relevant if your working life in academia is governed by the insanity of metrics – grant income, PhD students, impact, REf 4* ‘outputs’. You know it is insanity, so read on…..

James Wilsdon may as well not have inveighed against the ‘metric tide’, and Jo Johnson could have saved printers’ ink asking vice-chancellors not to waste academics’ time, and students’ fee money by operating multiple ‘mock’ REFs (BIS Green Paper November 2015 Chapter 2, para 7).

It is time for a critical conversation to take place about the use and abuse of metrics. In July 2015, Hefce published The Metric Tide, the report of a review body chaired by James Wilsdon, professor of science and democracy at the University of Sussex.

Despite the report’s chilling preface, announcing a “new barbarity” in our universities, we continue to witness the misuse of metrics as a tool of management in UK higher education. “Metrics hold real power: they are constitutive of values, identities and livelihoods”, wrote Wilsdon. Universities should proceed with caution, then, lest metrics should spread like a digital Himalayan Balsam and undermine the ethical architecture of universities.

It is ironic, but perhaps fortunate, that students find universities a very different experience than the academic staff who labour in them. For students, the intrusive scrutiny of metrics can at least claim to betoken a therapeutic and supportive institution. Generally speaking, the student ‘dashboard’ does not harbour the disciplinary function of its academic equivalent. 

For academic staff, audit has become a central organizing principle of life in universities (Strathern 2003). Our working lives are structured around the requirement to undergo ‘rituals of verification’ (Power 1997), and there are as many anticipatory audits as there are demands for post-hoc justification. Such is their prevalence that the behaviour of academics has been transformed so that they are interpolated primarily as auditees (Petersen and Davies 2010). Benchmarks, metrics and dashboards are examples of calculative practices (Shore and Wright 2015), used, apparently, to measure and improve the productivity of academics. This imposes a rationality whereby we face a future of ‘algorithmic regulation’ (Morozov 2014), and regimes in which employees are hierarchized according to metrics. University policy documents endeavour to justify these practices as essential, and even empowering to academics.

I have blogged previously about the difference between stewardship and agency approaches to performance management, citing a report from the Leadership Foundation for Higher Education (Franco-Santos et al. 2014).

Stewardship approaches frame a long term direction of travel which relies on individuals exercising autonomy, while agency approaches address short term goals via monitoring and tight control. While UK universities seem beholden to the short-term, agency approach, stewardship approaches are favoured by Umran Inan, President of Koc University in Turkey. He writes that “that any attempt to pass down norms or procedures” from on high is antithetical to creativity, and that universities must instead “allow unusual [and] inconsistent things to happen” (Parr 2014). At Koc, quality of scholarship is allowed to flourish and internal evaluations take place every five years. By comparison, in the UK, we are in danger of allowing academic freedom and creativity to founder under the distorting constraints of audit.

I am not fond of sports metaphors, but many vice-chancellors are. I understand that what has made the New Zealand All Blacks a great team is a sense that there is a long-term investment in each player’s development, rather than the England team’s reliance on a permanent sense of insecurity and enforced competition for their place on the team.

Raising the Bar is a sports metaphor that will be familiar to academics at Newcastle University, as they have become one of the latest universities to publish their expectations for research performance. All of this was initiated by managerial anxiety, amidst chatter about so-called ‘bottom Russellers,’ that Newcastle has been “lacking in competitiveness compared to other Russell Group institutions”. The Vice-Chancellor, Chris Brink stated in a ‘town hall meeting’ in November 2015 that Newcastle had lacked 4*-ness in the last REF, and that an institutional goal was to be in the top half of the Russell Group. This can be attributed purely to league table-induced status anxiety. But I do wonder when, exactly, did academia become a combat zone? Probably it was at the same time they started awarding stars, like US Army generals. When did the amount of grant money eclipse the actual content of the research? But Raising the Bar is a coercively innocent phrase. It conveniently conceals all the judgement, hostility, pain and pressure that we know will follow it. Academic endeavour is not something that can just be improved by order. Research functions within a context, an ethos and a dynamic.

Now that I have built a reputation for busting managerial myths about performance management, kind Twitterati send me their universities’ policy documents. As a linguist, I feast upon their discursively encoded ideologies. As a human being, a scholar and a friend of many victims, I weep at the cost for these individuals, but also for the future of universities.

This is my analysis of the document (see below): Newcastle University, Faculty of Humanities and Social Sciences, Research and Innovation Performance Expectations.

First of all, we find that these are expectations, not objectives, not targets, not goals. Expectations are much more finite and concrete, and do not permit that worrying prospect of slippage.

[T]he expectations on research active staff – so by definition if you do not meet them you are not research active, despite any other evidence to the contrary.

This document is focussed on research performance…..as this will determine our ranking in the next REF. And the key to this, we learn, will be increasing the number of 4* outputs. The parameters of ‘performance’ are drawn so rigidly, and amount solely to ‘being REF-able’.  This circumscribes any kind of professional autonomy, or even what should count as academic labour, guaranteeing that much of what academics do will be rendered invisible. What of the early career researcher, or graduate student celebrating their first scholarly publication? This will probably not be a 4*, and any pleasure or sense of fulfilment will be subdued.  How can any of us take pleasure in our work under these conditions?

 [W]e have largely relied on REF 2014 entry as a proxy for reaching the minimum expectations for research outputs. How is this possible? How would a local assessor know if an individual’s outputs were scored as the quoted minimum 3*? Individual REF scores are categorically not available; they have been destroyed (REF FAQ).

A criterion for a chair is: aspires to be in the Top quartile in UoA for income, or aspiring to 4* – how can everyone be in the top quartile? With success rates as low as 12%, then that is an expectation you will probably not meet. And how do you indicate aspiration, if you fall short? Are we now to be judged on the breadth of our imaginations?

Newcastle University is not alone in planning to audit their academic staff on attainment of quantifiable targets, with some, like grant capture, quite outside their control. Nor is the misery confined to the Russell Group; Newcastle joins a long list which now amounts to one in six UK universities, according to the Times Higher: Queens University Belfast, Imperial, Queen Mary University of London, Abertay University, Plymouth University, Robert Gordon University and the universities of Aberdeen, Dundee, East Anglia, Glasgow, Greenwich and Leeds.

If targets are unattainable, this leads to unmanageable levels of stress. This is objective setting 101. Employers have a legal obligation to conduct risk assessments and prevent known causes of stress. “Employers are only in breach of their duty if they have failed to take reasonable steps in the circumstances to prevent the stress”. Notice the word is ‘prevent,’ not alleviate stress. It is not helpful to offer mindfulness workshops, nor aromatherapy. The workplace should be managed in accordance with the law and decent moral conduct.

Of course, Chris Brink alluded to the fact that academics would be ‘supported’ in reaching the new bar; however the inevitable monitoring, reporting and surveillance will only serve to amplify the pressure of the audit. If we need evidence that targets and performance management cause insupportable stress, we should remember the tragic case of Stefan Grimm who took his own life after being threatened with performance management procedures at Imperial College. The coroner found Stefan’s death to be ‘needless’  and Imperial College said that ‘wider lessons’ would be learned. It is now very clear that the nation’s vice-chancellors have been unwilling to face the facts. There is systematic, mass victimization of UK academics.

And it is not confined to the UK; recent research from Australia on the impact of aggressive performance management on early career researchers (Petersen 2016 forthcoming) indicates that stress starts to manifest itself quickly, and has a negative impact on work. Many ECRs “struggled to articulate the value and worth of their work outside the productivity discourse” (2016:12). The constraints of metrics cause the content of the research to change, and researchers attempt to mirror what is ‘hot’ – likely to get funding under shifting priorities of research councils. And I have observed as well that we all tend to discount even solid scholarship – edited volumes or book reviews – if they are not REF-able. And in the same way we will learn to discount any work which is not judged – even by non-expert assessors – as worthy of a 3* or 4* ranking. As Petersen says of her informants, “they and the substance of their work become easier to control”.

Another control technique is to devalue the work of someone who is entirely dedicated to their scholarship, and whose whole identity is enmeshed with their work. Ammunition can easily be found when targets are so numerous and the scope so wide that almost every employee can be found wanting in some dimension. The targets reflect management’s construction of the ideal employee who is ‘compelled never to rest’ (Davies and Petersen 2005: 89). This offers employers the opportunity to apply policies capriciously: poor teaching scores may be overlooked in some cases, but lack of grant income is not. It is a licence for the academically insecure to settle grudges or academic jealousies with their more talented underlings. And so an increasing number of academics are being marched through performance management processes in which their unique contributions are rendered insignificant and their imagined lapses are deemed ‘incapability’. Targets must be attained at each and every period of audit. You may find that international esteem is never arrived at, even though your books – even translations of them – are still on the shelves. Was it achieved and documented in the last six months? That is all that matters. Professorship must be performed in these tightly delimited ways like a horse doing dressage. Threats of demotion are made. It is like rescinding Mark Spitz’s gold medals because he can no longer match Michael Phelps.

I have to question whether these quantifiable targets meet management’s declared aims of encouraging staff retention and offering a structure of support for academics. It is hard to imagine they could work positively in this way when, in some departments, significant numbers of academics are immobilized by emotional and professional breakdown.

In the fictions and contradiction of management Polari, this is returned to the apparently delinquent employee as ‘empowerment’, even as managers arrogate the power to press the detonate button on careers. This mendacious discourse serves to absolve the perpetrators of torture from the shattering incapacitation these procedures invoke.

The Vice-Chancellor of Newcastle University portrays the metrics of Raising the Bar as objectivity. It is not, nor is it objective setting; this is objectification. Let us remind ourselves of Martha Nussbaum’s (1995) original seven features of objectification:

• Instrumentality – to be treated as a tool for man’s purposes. At Newcastle, the function of an academic is to raise the bar, increase grant income and raise the university’s position in the league tables.
• Denial of autonomy – activity and what counts as work is tightly defined and controlled.
• Inertness – there are no human agents in the Newcastle documents. Grammatical subjects include ‘this document’, and “this aspect of our academic portfolio”, “a detailed analysis of the results” and “expectations”. There is the passive voice throughout, with just three instances of an unattributed pronoun ‘we’. ‘We’ is an inherently ambiguous pronoun. It can be used either inclusively, or exclusively of the addressee. Looking at the contexts: “we do not expect all staff to have equal strengths”; “we have largely relied on REF 2014”; “We will take early career researcher’ – these usages seem to retain the prerogative of its exclusive attribution.
• Fungibility – interchangeability with other objects of the same type. How often have you heard managers say “we have an open door policy”? And notice that in HR-speak, there are no people, with contributions to make, there are only ‘roles’, and these can have the status of vacant, or filled.
• Violability – something that can be broken, violated, smashed into. Many academics are now looking at ruined careers and broken ambitions. We have ensured that an academic career has become unsustainable in the long term.
• Ownership – something that can be traded or commodified; we now hear about ‘a transfer market’ for ‘4* 4’ professors – professors with four 4* outputs.
• Denial of subjectivity – your feelings need not be taken into account. Indeed, there is no way of expressing them, as we are forced to account for our work in terms of management-defined metrics. This is known as illocutionary silencing (Meyerhoff 2004).

Does this seem familiar? Objectification is the reality behind what we are supposed to believe is ‘empowerment’, a word now taking that slide into the semantic inversion prefigured by ‘excellence’ and ‘quality’. But generations of women have been persuaded that objectification is empowering, so why not try it out on academics?

This is a job creation scheme for HR as more and more academics will find themselves falling short of ‘Raising the Bar’. If we are lucky, they may embroil universities in grievances and libel suits which will chasten managers’ love of ‘robust’ metrical solutions to problems which do not exist. Vice-chancellors use a dip in metrics, or the effects of incapacitation – ‘falling over’ – in their jargon, as an occasion to axe courses, subject, departments and schools. We are about to enter an era of manufactured instability in universities in both staff turnover and academic offering.  

Metrics, then, are unlikely to offer any of the certainties that their champions have promised, and doubly so because of the sheer irrationality that governs their application. The bar must be raised, and raised again. No-one must slip beneath the bar. There is only the bar, the metric that cannot lie. Except it does. There is always a rush to judgement as metrics occlude any other evidence. This is the weak spot, and one that offers a route to resistance. What about content? What about the imagination, passion and risk-taking that animate research? What about bright people having fortuitous conversations?

There has been a hollowing out of the sense of purpose of universities as power is skewed towards the managerial function. Despite vice-chancellors’ assurances that There is No Alternative, it doesn’t have to be like this, and it is our job to take every opportunity to make this clear. Good work cannot be sustained under these conditions of pressure and surveillance. Academics are not servo systems whose online functioning can be monitored and tweaked in response to new demands. Consider this reality; academics with all their strengths and imperfections are the people who attract students. I gift several Saturdays a year to my institution because, it seems, I am able to persuade students to study there. Nobody ever signed up because of the award-winning Human Resources team, or was swayed by the robustness of the Improving Performance Procedure.

So let’s indeed raise the bar. Let’s raise the bar for decency, humanity, respect and trust. Let’s realise that academic staff do not have either the resources or the capacity to keep expanding their workloads and output every year, and please let’s keep in mind the human consequences of systems that push people above, over and beyond. And let’s return to that meaning of ‘we’ and allow ourselves to feel that it includes everyone who works in a university and not allow it to pertain exclusively to management. I hope that, perhaps, one day, vice-chancellors and their senior management teams will wake up and remember they work for universities; they are not the university.

 Video of talk at Newcastle University 25^th November 2015

Newcastle Humanities and Social Sciences RTB doc

 

References

 Davies, B. and Petersen, E.B. 2005. Neo-liberal discourse in the academy: The forestalling of (collective) resistance. Learning and Teaching in the Social Sciences 2 (2): 77-98.

Department for Business, Innovation and Skills. 2015.  Fulfilling our Potential: Teaching Excellence, Social Mobility and Student Choice. November 2015. HMSO.  https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/474227/BIS-15-623-fulfilling-our-potential-teaching-excellence-social-mobility-and-student-choice.pdf

Franco-Santos, M., Rivera, P. and Bourne, M. 2014. http://www.som.cranfield.ac.uk/som/dinamic-content/news/documents/PerformanceManagementinUKHigherEducationInstitutions.pdf

Meyerhoff, Miriam. 2004. Doing and saying: some words on women’s silence. In R.T. Lakoff. Language and Women’s Place: Text and Commentaries (revised and expanded). M. Bucholtz (ed.). Oxford: Oxford University Press. 209-215.

Morozov, E. 2014. To Save Everything, Click Here. Penguin.

Nussbaum, Martha. 1995. Objectification, Philosophy and Public Affairs, 24 (4), pp. 249–291.

Parr, Chris. 2014. Young universities’ secrets of success, Times Higher. July 17^th.

Power, Michael. 1997. The Audit Society: Rituals of Verification. Oxford University Press, Oxford, 1997.

Petersen, E.B. (forthcoming, 2016) The impact of managerial performance frameworks on research activities among Australian early career researchers, in ed. K. Trimmer Political Pressures on Educational and Social Research. NY: Routledge

Petersen, E.B. and Davies, B. 2010. In/Difference in the neoliberalised university.  Learning and Teaching in the Social Sciences 3 (2): 92-109.

Shore, C. and Wright, S. 2015. Audit culture revisited. Current Anthropology, 56 (3): 421-444.

Strathern, M. (ed) 2000. Audit Cultures: Anthropological Studies in Accountability, Ethics and the Academy. London: Routledge.

Wilsdon, J., et al. (2015). The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management. DOI: 10.13140/RG.2.1.4929.1363