TEF metrics: proxy, toxic and idiotic…but very neoliberal

This is an updated version of a previous blog post and apeared recently on the LSE Impact blog. It also draws on my article, The Accident of Accessibility: How the Data of the TEF Creates Neoliberal Subjects, published in Social Epistemology and appeared in a special issue on Neoliberalism, Technocracy and Higher Education; Guest Edited by Justin Cruickshank and Ross Abbinnett.

The stated aim of the Teaching Excellence Framework (TEF) is to encourage excellence in teaching in higher education and to provide information for students to make improved decisions about the courses they take at university. In this post, I argue that contrary to these goals, the TEF is only marginally interested in teaching quality and instead contributes to the increasingly personalised and transactional nature of university education, provides perverse incentives for educators and ultimately positions participants in higher education as neoliberal subjects.

The stress occasioned by constant demands for academics to submit to evaluation has been well documented (Loveday 2018Morrish 2019), nevertheless the bureaucratic appetite for surveillance is fed by the ease of accessing quantifiable data. This blog looks at the repercussions for universities, students and academics of the chosen metrics of the Teaching Excellence Framework outlined below.

In 2016, the government published a White Paper on plans to reform higher education, Success as a Knowledge Economy (SKE). The brainchild of Jo Johnson, former minister for universities, it laid out a justification for the introduction of the TEF as the solution to a series of imagined faults with universities, such as ‘lamentable’ teaching, and a lack of ‘return on investment’ for graduates of some university courses. Johnson promised that it would never become ‘big, bossy and bureaucratic‘.

The selection of metrics for the TEF appeared arbitrary: student satisfaction with teaching, assessment and feedback (taken from the National Student Survey, NSS), continuation (retention) rates (taken from the Higher Education Statistics Agency, HESA) and a measurement known as Longitudinal Educational Outcomes data (LEO). These proxy measures have nothing to do with classroom teaching, and yet have become imbued with spurious validity.

David Beer writes that metrics measure us in new and powerful ways such that they order the social world and shape our lives. Whereas, metrics can appear neutral and necessary, in the case of TEF proxy metrics, we can argue that their whole purpose is dirigiste. Since the introduction of tuition fees in 1999, successive governments of all hues have sought to reconfigure universities as instruments of market ideology.

Each of the TEF proxy metrics underpins this strategy: the NSS arose from the project to transform students into consumers; the retention rate metric was similarly designed to measure consumer appeal, even though it reflects more accurately the social advantage of the student body; LEO data became available with the passing of the Small Business, Enterprise and Employment Act. This legislation permits mining of individual tax records for integration with student loan and university degree records and enables the government to assess which courses produce graduates most able to repay their tuition fee loans. It becomes apparent, in the context of some of Jo Johnson’s remarks (20152016a2016b), that the role of the TEF was to shift student choice towards STEM subjects which, on average, lead to higher-paying careers, which the government judges to be more beneficial to the economy.

These metrics are not simply benign and have serious implications for academics whose labour is being evaluated along new and unforeseen dimensions. For instance, they must now privilege the potential popularity of a subject before proposing to teach it. They must justify both new and current courses on the grounds of ‘employability’. And so, innovation in teaching and research is determined by appeals to economic value and the capricious choices of 18-year olds, rather than by the advance of knowledge or professional judgement of academics. Nevertheless, universities which fail to score highly on the TEF metrics risk being deemed ‘failing’ and the government may seek to pressure them to close courses which do not deliver the right ‘outcomes’. These market reforms were skillfully packaged as enhancing student ‘choice’. Furthermore, market ideology and tenets of neoliberalism such as competition, choice and individual responsibility are all bolstered discursively throughout the SKE White paper and by repetition by politicians.

A neoliberal discourse 

Significantly, the government avoids the word ‘university’, insisting that that there has been insufficient penetration of ‘the market’ in the higher education sector, and so in order to instil ‘competition’ and ‘choice’, the market must be opened up to new providersProvider also implies a transactional function for universities – a kind of cash for credentials scenario in which the institution simply provides its product to its ‘customers’ in exchange for money.

Investment, and its cousin, return on investment (ROI), are further linked discursively with teaching quality, in a way that implies that receiving information about the latter would in turn ensure a guarantee of the former: ​“The quality of teaching should be among the key drivers of a prospective student’s investment” ​(SKE Ch 2 para 8 p43). However, evidence reveals that there is little material correlation. Reports by the Institute for Fiscal Studies analyse the government’s Longitudinal Earnings Outcome data (Bellfield and Britton 2018Britton et al. 2016) and show the outcomes-require-competition assumption to be a myth. The best predictors for graduate earnings (the likely meaning of investment here) is found to be parental income and prior attainment. Return on investment in this way has arguably been used deliberately to reposition higher education as a private, rather than a public good.

Although the government has claimed the TEF will empower students, the Office for Students,  which was supposed to  ensure that student choice is placed at the centre of university policy making, fell silent when a student voice was raised in challenge. In 2017, a great deal of manufactured outrage was expressed in right-wing newspapers when a University of Cambridge student, Lola Olufemi, expressed a request for the English department to more fully decolonise the curriculum. Nor has recent higher education legislation or market discipline enabled a broadening of student choice when we consider that, since the implementation of the Browne Review funding system of tuition fees, there has been a precipitous decline in the availability of part-time routes for undergraduate study.

Conclusion

The accident of accessibility of particular metrics, inasmuch as they overlap with neoliberal priorities, has therefore determined which data will serve as Key Performance Indicators in the TEF. What may seem to be a random assortment of proxy data points, has in fact  served an agenda to refashion universities, staff and students as neoliberal subjects. The TEF audit therefore only appears to be tangentially concerned with quality. Instead it incentivizes universities towards a more pronounced concentration of business and science curricular provision and is reinforced via discourse which all participants in the framework are compelled to cite.

There is a discussion page for articles in the special issue of Social Epistemology which readers can find here.

Leave a comment