Category Archives: HE policy

The accident of accessibility: How the data of the TEF creates neoliberal subjects

This is the link to a video of a talk I gave to the Digital University in a Neoliberal Age Symposium, organised by the Contemporary Philosophy of Technology Research Group. Title: The accident of accessibility: How the data of the TEF creates neoliberal subjects.

Abstract:

In an era of neoliberal reforms, academics in UK universities have become increasingly enmeshed in audit, particularly of research ‘outputs’ via the Research Excellence Framework (REF). A new Teaching Excellent Framework (TEF) has emerged in 2017, whose results are determined primarily by proxy data of National Student Survey (NSS) scores, retention data and Longitudinal Educational Outcomes data (LEO), i.e. salaries of graduates. This has been made possible by SBEE (Small Business, Enterprise and Employment Act 2015) legislation which has enabled data mining and synthesis of data streams from records held by the Student Loans Company (SLC), HMRC and universities themselves.

These two audit processes, REF and TEF, were originally envisaged as instruments to evaluate research and teaching, respectively, at institutional level. This had a distinctively neoliberal purpose in seeking to mould universities more closely towards serving the economic needs of the nation. The REF, however, has also been recruited as an instrument of individual performance management in universities, with each academic forced to compete in academic output and research funds with the most talented and unencumbered scholars. The TEF, similarly, bestows an institutional ranking, but will rapidly be repurposed in order to shape the behaviour and priorities of academics. For example, the participation of local areas (POLAR) classification allows universities to be rated according to their success against the Widening Participation (WP) agenda. In this way, universities can appear to fail by revealing larger differential outcomes for target groups according to ethnicity and social class than their benchmark permits. The discourse of the TEF legislation, bolstered by studies from HEA/HEPI, assumes the source of inequality of outcome is poor teaching and requires corrective action by universities. Further justification for surveillance and quasi-regulation is borne by appeals to ‘value for money’ and ‘competition’. Universities are positioned as subject to market forces, and students positioned as consumers. Universities are responding by creating ‘managers for the student experience’ whose responsibility it will be to oversee change, without ever addressing the question of what causes differential outcomes, or what actions on behalf of government or institutions might make a difference.

I argue that what seems to be an arbitrary constellation of proxy data points has in fact been a calculated plan to render universities, staff and students as neoliberal subjects. The accident of accessibility, inasmuch as it overlaps with the neoliberal imperative, has determined which data shall function as Key Performance Indicators (KPIs). These KPIs are signalled via metrics-driven student and staff dashboards which offer no retreat from the interpretation and coding imposed by government, and the whole assemblage is cemented by discursive choices which align with neoliberal principles. In this way, the ideological purpose of the legislation and the audit is realized: the imposition of institutional and personal responsibility for structural inequality has been achieved.

The Government White Paper Success as a Knowledge Economy, May 2016, will form the text for corpus analysis of keywords, discourses and metaphors.

Advertisements

When does Prevent prevent freedom of speech?

The piece first appeared in Research Professional online forum, 19th September 2017. 

Earlier this year I was expecting to deliver a talk to a UCU meeting called to oppose the University of Warwick management’s proposal to reform Statute 24. This refers to amendments to the University’s procedures for Disciplinary, Grievance, Redundancy and Removal for Incapacity on Medical Grounds for Academic Staff. I had been asked seven days previously to address the meeting, but because Warwick has a requirement to give three weeks’ notice for approval of a visiting speaker, the organisers felt the meeting was unable to proceed.

Warwick’s ‘three week rule’ is one of the procedures adopted by the university under the government’s Prevent agenda.  I was invited to an emergency meeting and the union organisers needed to find a speaker fast. Understandably, they had other things on their minds and forgot to apply for speaker approval. Warwick administrators quickly responded to say that they had received no application for speaker approval, and indeed would not have refused any application on my behalf. There is, apparently, a procedure which covers the eventuality which occurred:

The principal organiser must ensure sufficient time for the HoD or nominee to give consideration to any concerns, and for the University to review the request should the HoD or nominee deem this necessary. If so, and where possible, the University should be notified of the speaker request in question at least three weeks prior to the event, to enable a full risk assessment to be conducted and any mitigating arrangements to be put in place. If it is not possible to provide three weeks’ notice, the department should inform the University as soon as practicable.

Perhaps staff were not well informed about this.

Like most academics, I do a fair amount of speaking at conferences and events, and this was my first encounter with Prevent or any need to get prior approval before appearing on campus. A three week notice period for all speakers seemed to me rather excessive. However inadvertent, the resulting cancellation acted as an impediment to the free flow of discussion, and has hindered organisation of a trade union activity.

Is there scope within Prevent to do things differently? I decided to do a bit of research. I started with the HM Government Prevent Duty Guidance for Higher Education.

When deciding whether or not to host a particular speaker, RHEBs should consider carefully whether the views being expressed, or likely to be expressed, constitute extremist views that risk drawing people into terrorism or are shared by terrorist groups.

RHEB – Relevant Higher Education Bodies. The government would always rather use an exclusionary acronym than call them universities, but this was a new one on me. Mind you, this was written in 2015, before we had to start calling them ‘providers’. The document contained a lot of information about defining terrorism, assessing risk and forming action plans, but no specific guidance on implementation. That advice is provided in a 44 page document from Universities UK on external speakers in higher education institutions.

There is a flow chart for speaker approval, and guidance on best practice for communicating the policy and timeline. Nowhere is a review period of three weeks suggested.

Some of the advice contained in this document, however, is almost guaranteed to impinge on academic freedom. For instance page 20:

Who is chairing the meeting? Are they sufficiently qualified to provide balance and challenge during the event? What is their stance on the topic under discussion and is this likely to impact the smooth running of the event?

Will hosting the speaker have reputational risks for the institution? Is the event likely to attract media attention and if so how can the university manage this effectively?

In my case, since I had been asked to speak in opposition to a proposal by the university’s management, this might have been unsympathetically construed as posing a reputational risk for the university. So the answers to the above questions, and the assessment of risk, may reflect the assessor’s stance and position in the institutional hierarchy.

An interesting issue arises on page 32 in that academic freedom does not necessarily apply to visiting speakers. We learn this is because academic freedom pertains only to teaching within a university:

The legal basis for academic freedom focuses on the teaching activities of staff and the freedom of institutions and their staff to determine admission criteria and the content of courses. Beyond the freedom of speech provisions, the legal framework does not extend academic freedom to the activities of visiting speakers.

And so to the local implementation of the policy at Warwick. The Warwick Prevent Action plan (07/11/2016) says:

Regulation 29 has been completely rearticulated to better foreground the University’s commitment to Freedom of Speech. The Regulation is complemented by a suite of supporting procedures for the approval of external speakers. This includes a light-touch approach for academic visitors and for external speakers associated with commercial conferences.

Regulation 29 says very little about freedom of speech; it has rather more to say about risk assessment and obligations to adhere to guidelines on university branding.

Another document outlines procedures for the approval of external speakers for Students’ Union, Student-Led, and Institutional-Level Events.

Organisers of events involving external speakers encompassed by these obligations, must complete and submit the External Speaker Request Form at least three weeks prior to the event taking place. The event must not be confirmed with the guest speaker until approval has been received from the University or the Students’ Union.

A requirement to give three weeks’ notice does not seem like ‘light touch’, but perhaps local culture permits the rule’s uneven application. I attended a conference at Warwick this year at which a speaker was thanked for stepping in at the last minute. Nevertheless, if you are a union official, you might be deterred by the suspicion that you could be under greater scrutiny.  Clearly my union hosts were cautious about getting their fingers burned, and that was enough to deny a union meeting a knowledgeable speaker.

My experience, and a swift survey of some available university policies, tells me that the Prevent agenda applies with differing degrees of scrupulosity across the sector. Despite Warwick’s wish to appear ‘light touch’, their procedure is lengthy and entails a risk assessment which must apparently be conducted by management. There are other institutions whose policies merely ask that organisers undertake their own risk assessment, and then ‘escalate’ to management if there is a reason for concern.

The University of Bristol policy states:

The Event Organiser (the person responsible for the event) must undertake a self-assessment (using the questions in section 2) to determine whether further scrutiny and support from the University are required. If the Event Organiser reasonably decides that there are no issues, the event can go ahead. It is anticipated that the vast majority of events organised will fall into this category.

Commendably, Nottingham Trent University does not distinguish between internal and external speakers and also allows organisers to make their own assessment of risk:

Formal approval from the University must be obtained, in advance, for any event to be held on the University’s premises (whether or not an external speaker is involved) where it is expected, or reasonably foreseeable, that the event will raise controversial issues which may risk infringement of or non-compliance with the University’s Code of Practice on Freedom of Speech.

These are preferable models to follow as they signify a high degree of trust in the judgement of members of the university. This is appropriate for learned, intelligent, responsible scholars and I can’t imagine how a vigorous research culture can thrive without this. Given that universities are currently under the cosh again on the issue of academic freedom, perhaps they can be encouraged to give some thought to reforming procedures which unnecessarily curb the freedom of all speakers to spontaneously address issues as they arise.

A day later and this thoughtful piece appeared from Smita Jamdar on the tensions between the government’s insistance on Prevent, and new regulatory powers on freedom of speech on university campuses.

 

 

Managing malice, reaffirming Robbins

My last blog post addressed some of the misconceptions surrounding the award of first class degrees in UK universities. I was favorably surprised by the reception and the number of people who seemed relieved to be presented with another explanation than the now customary media allegations of dumbing down, declining standards and apathetic students being rewarded for sub-standard work. However, in the print media, there swiftly followed another set of familiar accusations  – that universities are admitting students with poor A levels to under-subscribed degrees.  Even more concerning, apparently these students won’t be able to secure a graduate job commensurate with their knowledge and skills.

This year’s cohort of freshers will experience uncertainty over value for money, having heard that universities are replacing credentialed academics with graduate teaching assistants or disenchanted teaching fellows.   Even lecturers with permanent posts are bailing out with stress-related illnesses.  Many students will have internalized the suspicion that a share of their £9000 could be gifted to an avaricious vice chancellor by a secretive remuneration committee. And they harbour the nagging thought that if they defer admission for a year or two, they might not have to pay such exorbitant fees.  And now, even if they overlook this catalogue of condemnation, students are told that they may be denied the prize of a first class honours degree, as universities could be punished if they award too many of them. And in any case, even the Guardian believes that academic judgement of degree classifications is so flawed it amounts to universities ‘marking their own homework’, and questions whether a first from one institution is equivalent to one from another. 

As I write, it is encouraging to know that for the first time in a long time, there has been an immediate and robust response to some of the critics from the new CEO of Universities UK, Alistair Jarvis, citing “misinformation, muddled argument and even a little malicious intent.” It is a welcome intervention.  

The uniting theme to all these accusations is that universities are doing a poor job and standards are slipping. The tired old argument goes, that if only 5% of students were admitted to universities in 1970, and today 32.5%, then it stands to reason that the bar must have been lowered. I suggest such critics go and read this excellent blog by Mike Ratcliffe, More Means Better, which is aimed at deflating the fiction that higher education should be the provenance of a self-defined elite.

My contribution to the debate has been to rebut the argument that universities have no check on standards and quality of teaching and assessment. My last blog piece pointed out that student achievement at university is criterion referenced, i.e. if a student reaches a threshold of learning, they will be awarded the same mark as every other student at this level of achievement, regardless of how many of them there are. Every student’s work is blind-marked, moderated (double checked) by another qualified academic, and passed before an external examiner whose job it is to monitor standards and comparability. This is fair to both students, and institutions who need to safeguard standards and reputation. At present, the reputation of UK universities is high – outside of the UK. It appears to be a peculiarity of UK journalists that they condemn the system that educated them. The fact that so many who voice skepticism are graduates of Oxford University has not escaped me, but you cannot be sure whether their views are the product of careless ignorance or exclusionary elitism. We can probably assume, though, that they will foster the ascent of their own children to suitably selective universities.

Let me offer an analogy which might clarify why ‘more means worse’ arguments are unsound, and which sheds some light on objective criterion referencing and improvements in performance.  

Recently untethered from the academy, I have been able to spend time during the summer with New York Open Water – a group which organizes marathon swims in New York. The most arduous of these swims is the 8 Bridges Hudson River marathon swim. It is a multi-stage swim, and each day’s distance of between 13.2and 19.8 miles must be completed within the time window allowed by the tidal flow of the Hudson River. It is preposterously hard to swim a marathon on consecutive days over a week. Between 2011 and 2016 only six swimmers managed to successfully accomplish all seven stages of this 120 mile swim between Kingston and New York Harbor, and there were two years when nobody made it. Nevertheless, the 2017 cohort furnished us with nine new entrants to the 8 Bridges hall of fame. Could anybody seriously suggest that somehow the standard required for success has slipped? Quality control tanked? Demand an investigation from the regulatory body? The 2017 event took place over the same course, had the same organizers and was run under the same rules of marathon swimming.

So what might account for the sudden and massive increase in success? Firstly, the profile of this event has risen across the international community of marathon swimmers, so many more people see it as an accomplishment they might wish to add to their CVs. Secondly, techniques of training, nutrition, mental and physical preparation have been customized towards the requirements of this event. It adds up to a highly targeted approach to this demanding swim, and a consequent rise in the success rate has been the result.

The analogy with university degrees is this; contrary to the misgivings of the media critics, universities have become much more focused on how to teach students in interesting and varied ways, on how students learn, and how to embed feedback and progression into the assessment process. Every student is made aware of the requirements of the course, the learning outcomes and assessment criteria. No surprise, then, that they focus their efforts on meeting these.

It is different world from the exams encountered by students fifty years ago. Then, nobody thought it off limits to set questions on material the course had not actually covered. Arts and humanities courses, particularly, saw their final exams as an assessment of general erudition. I have seen a finals paper from Bristol university (circa 1965) sat by all arts students which assumed knowledge of literature, poetry, artistic movements as well as music. There was no preparatory taught course; students were just expected to have absorbed this knowledge as part of their autonomous intellectual development. The Robbins principle had, after all, stated that the role of universities was to produce “cultivated men and women; and to transmit a common culture and common standards of citizenship.” 

It would be facile to compare exam papers from the 1960s with those of today and declare the latter less challenging. In the intervening years, the emphasis has swung from familiarity with high culture, to the acquisition of transferable skills and intellectual agility. We now appear to be moving beyond this to a requirement for demonstrable ‘learning gain’ in graduates. 

Failure to understand the evolution of teaching and learning in universities may be at the root of some of the dismissive articles we have seen over the last year. It sometimes happens that the products of elite institutions believe their own myths. When you are told you are very ‘bright’, and are surrounded by others who are deemed ‘bright’, you tend to believe in your own exceptionalism and entitlement, when in fact ‘bright’ is often nothing more than the expression of privilege and social capital. As an educator of over 30 years, I am unimpressed by ‘bright’ because it has always seemed one of the least useful predictors of success in higher education, or indeed life. Give me the curious, the challenging, the creative, the hardworking, but above all the persistent. And a measure of self-belief is always helpful. These are the attributes I observed among the marathon swimmers I met, and among the students who distinguished themselves by improving year on year. So rather than objecting that universities are somehow diluting their standards and bestowing worthless degrees, let’s at least acknowledge two important legacies. Students are emerging from school, motivated, qualified with A levels and fully eligible for higher education. This is also one of the outcomes envisaged by Lord Robbins who argued that university places “should be available to all who were qualified for them by ability and attainment” Robbins believed that many more students could benefit from university than had been able to access it prior to 1963. He went to great lengths to provide statistical evidence that more would not mean worse in terms of lowering of standards, and indeed the expansion of UK universities was matched by enhanced international standing.

The Robbins’ principle of democratization of universities was what we wished for the nation’s young people in 1963 and it is worth defending today. It is what our politicians and journalists want for their children, even as they undermine those advantages for everyone else’s.

The Disrespect of the TEF

 

I have been a few days late to the sector-wide freakshow that is the TEF results.  There has been a news-grabbing but probably temporary perturbation to the traditional hierarchy of mostly English universities, their VCs enlivened by the prospect of being able to raise tuition fees. The resulting categorisations of Gold, Silver and Bronze brought forth just one expression of outrage from Sir Christopher Snowden, the VC of the University of Southampton, and former Universities UK president. Altogether three more (Liverpool, Durham and York) have joined him in launching an appeal. Endorsed by the vice-president of the NUS, Sorana Vieru, Snowden levelled this criticism, “I know I am not alone in having deep concerns about its subjective assessment, its lack of transparency, and with different benchmarks for each institution removing any sense of equity and equality of assessment.”

The statement would have had more force if it had been delivered before the data for the TEF had been sent to Hefce by compliant vice-chancellors. Better still, the sector could have prevented needless and undeserved reputational damage to a majority of universities which received Bronze and Silver ratings if they had stuck together and paid heed to the uncontestable arguments against the TEF. The Council for the Defence of British Universities (CDBU) has published several informative pieces on their website. A key contributor is Professor Dorothy Bishop of the University of Oxford whose argument is summarised in this excellent reply to an article by Edward Peck, VC of Nottingham Trent University.

Her key points are that the justification for the TEF of supposed ‘lamentable teaching’ is unfounded, and in any case, weaknesses in teaching can be diagnosed and addressed by the current QAA inspection and quality framework. Crucially, “The validity of the National Student Survey as a measure of teaching quality has been roundly criticised, and these criticisms appear to have been accepted by the chair of the TEF.” Also, “the statistical properties of NSS data have been described as unsuitable as a measure of teaching quality by the Office for National Statistics, the Royal Statistical Society and, most recently, by Lord Lipsey, chair of the All Party Parliamentary Group on Statistics.”

This should have been a damning enough condemnation to stop the TEF in its tracks before the HEAR Bill could be hustled through parliament before the General Election. It is disturbing that a process which would not get through the first pass of peer review if submitted to an academic journal is now being triumphantly lauded as begetter of a new and ‘disruptive’ hierarchy of universities which better reflects and serves ‘student choice’.

The majority of those who teach in universities are clear that the TEF methodology is fatally flawed, and the results meaningless. There are those in government and the media, though, who still defend its meager claims on credibility. One such is Nick Hillman, Director of HEPI, the Higher Education Policy Institute, a position he assumed after a spell as the Special Advisor to David Willetts when the latter was Minister of State for Higher Education. Nick appears, from my many years of interaction with him on Twitter, to be an amiable and fair minded human being. I appreciate his willingness to engage with critics of the TEF and higher education policy generally. However, I was alarmed at some comments made during this thread. It is worth showing the interaction in full below. The thread starts with the CDBU taking issue with Nick’s view that the TEF offers students important information on which to base their choice of university, a view which echoes the wording of the HE White Paper of May 2016.

TEF 1

TEF 2

TEF 3

TEF 4

TEF 5

What surprised me was Nick’s response to my suggestion that an ill-conceived array of proxy metrics might not serve anybody’s purpose. He seems to believe that it is too much to ask that any policy should work well from Day 1. I suggest that a well-recognised business process, Six Sigma, which focusses on identifying flaws and process improvement, could be adopted here. In an era when public sector organisations are directed to be more business-like in terms of efficiency and effectiveness, I would have thought this would be an article of faith in policy analysis too. Clearly, getting it right in government circles is too much to ask, as Nick reflects on “the messiness of policymaking in a democracy”. Actually, you don’t need anything as systematic as Six Sigma, you just need evidence-based policymaking, and it starts with paying attention to well-informed critics like Dorothy Bishop.

My point in writing this is not to make a personal attack on Nick, who, as I said above, distinguishes himself by his willingness to engage with alternative viewpoints. This is rather a howl of despair at an apparent double standard in expectations between government and those who must deliver their policies. What I am hoping to convey is the frustration of academics who are asked to restrain their own exploratory instincts in favour of a highly-regulated, audited and disciplined approach to working which has predictability and safety as its guiding principles.

What an ironic reversal this is. In previous years we might have expected universities to have license to experiment, to fail, to be messy in their approach. This was how discovery and progress was assumed to move forward. No longer, apparently. While government is afforded the indulgence of failure in policy-making, this is not the case for those who are charged with delivery. For policy makers there appears to be no obligation to embody ‘excellence’, to do what works, to get it right first time, to be evidence- and outcomes-led. These are instead demanded from those who must fulfil them under conditions of an increasingly demanding workload into which more government policy initiatives are emptied on a continuous basis. Unlike government, there is no five-year window for memory slippage when you are subject to performance improvement procedures every three months over some imagined lapse in ‘excellence’. It must be an enjoyable life, driving policy changes when you know that ‘messy’ outcomes will be tolerated, but it is a rather different story in academia 2017.

Let me give just a couple of examples. Research can be a step into unknown territory, but in REF culture, it would be an unwise scholar who set out without a clear sense of – not just results – but the ‘value’ and ‘impact’ of their work. No unpredictable outcomes possible. And similarly, academic staff are expected to fund their research with grant money from research councils with success rates for applications as low as 11%. Similarly, in our teaching, despite recognising that each individual student will take something different from our courses, we must submit them to a system of empty standardized ‘learning outcomes’. And when we have assessed them in a way which belies their imagination and intellectual response, we must endeavor to portray this as ‘personalisation of the learning experience’. There is little tolerance for even necessary ‘messiness’ in academic life. We mourn its passing.

Nick has urged universities to offer more support for students with mental health difficulties in a recent HEPI reportI look forward to another HEPI report which considers the crisis in academic staff mental health, and the role of frameworks such as the TEF and REF in heightening this. They may seem like benign instruments of audit, necessary to justify the considerable public spend in higher education. This would be uncontroversial but for the fact that these have been folded into the disciplinary mechanisms of New Public Management in universities. They too have been personalized and dashboarded into instruments of performance management.  And if HEPI does join the growing band of voices advocating for a more humane university workplace, I hope the report encounters a more gracious reception than I did when I spoke truth to power.

Nick seems to find my retort disrespectful, and if I have been, I offer my apologies. I have always said that the only thing that ever trickles down is contempt, and academics feel it raining down from government, magnified by sections of university management.  As I indicted in my response, democracy has perished in universities alongside ‘messiness’. And when such a double standard is in place, and you are only as good as your last ‘win’, even in the face of structural obstacles, it is nothing less than abusive.

Resist, Insist, Persist

One of the things I miss about my life in a university is the recommended reflective feedback sessions with my managers after each staff development training. I’m sure they don’t feel the same. But to continue the habit of a lifetime – and with very sincere gratitude – I want to thank everyone who read this piece I wrote in the Times Higher (2nd March 2017)  Thanks especially to those who commented, retweeted, quoted, DM-ed, emailed or hollered. Reactions fell into largely three categories: congratulation, commiseration and corroboration.

First the congratulations, and there were some lovely QTs which featured adjectives like inspirational, insightful, courageous, powerful, excoriating, remarkable, brilliant. These all made me blush, but these linguistic judgments of esteem and veracity fuelled me with more determination to keep writing.

From the US, a Professor Emerita wrote: “It’s very interesting to read the comments of many here, who speculate on how to rebuild or create a better system, and whether or not that is possible. With such minds as I see among you and your friends, dear Liz, there seems to be a great deal of possibility–and it’s exciting to imagine what very different ways of learning you may bring into being in the coming years”.

From a fellow blogger with a keen critique of university policy and implementation:  “You’ve become a lodestone for us all as an example of the ethic of academia, and how difficult it’s becoming to behave ethical in the current structures. You’ve helped me personally and intellectually Liz – I hope there’s a lot more to come”.

Other colleagues offered commiseration and expressed their own sense of disenchantment.

This was from a union colleague: “It is getting increasingly difficult for me to experience the constant trampling of basic professional ethics. I was disgusted to hear what had happened to you, Liz, after sharing your views on performance management. Sadly so few academics today understand the value of academic freedom, in part because they are not doing work sufficiently controversial as to require its protection”.

It was consoling to hear from a few pro-vice chancellors. One wrote: ” Very sorry to hear about your recent experiences. You seem to have been treated very badly. Your Times Higher article is really effective in keeping these issues on the agenda”.

From another: “When I read your article in the THE this morning I was overwhelmed by a great sadness to think that you had left your university following an absurd ‘disciplinary’ process.  This evening I read it again and I am furious to imagine what nonsense you must have been through.  I am just sorry I did not know about it while the process was in train, to provide solidarity and counsel.  This injustice will remain a running sore as long as it is acceptable to think that the best way to do academic work is outside the academy.  This wrong must be righted.  It is just such a pity that academics and our union (don’t make me laugh) is so bad a mobilising around truly important issues”.

Two readers told me privately that they had broken down in tears after reading my piece. When a person suffers burnout and emotional distress, their own empathic reactions to another’s plight can be overwhelming. This I know from experience.

Others corroborated my analysis by sharing their own experiences of audit culture in universities. The panoptical nature of the surveillance, the punitive actions that accompany it, and the often unattainable targets demanded, all add up to stress, despondency and mental illness. One colleague pointed towards a future of algorithmic performance management of the sort identified at Amazon in the New York Times expose.

“Here at [Russell Group University] we have [XX company] coming in soon which allows real time micro level performance management via ‘dashboards’ recording all data on all staff for the duration of the ‘staff member’s life cycle’…. I guess that means electronic module evaluation feedback to save processing time, being added to H-index etc. data. Although this is denied by managers, part of the purpose with this is to strip out middle management and allow central / senior management to set targets for ‘teams’ and saying they can ‘liaise’ with the ‘core member’ if they need resources to achieve their targets, which really means, ‘we have direct control, total ‘transparency’ and can get rid of teams that ask for too much”.

A colleague in Australia DM-ed: “I’ve just come out of a traumatic couple of weeks in which I was asked to write a self-evaluation report identifying and quantifying my value to the university. I’ve been told that unless I can come up with some ‘low-hanging fruit’ in the short term my days are numbered etc. And I am – by any measure – a highly productive academic with millions of dollars in grant money, a plethora of publications etc”.

A colleague at another Russell Group University wrote: “The problem seems to me to be that the institution’s demands for compliance wreck our intellects (and our resolve and resilience), while stamping on us with disciplinary power whenever we point this out”.

This theme was best summarized in a tweet by one respected commentator (well, I respect him, and if he’s reading, I’d be really chuffed if he’d follow me on Twitter. Like what else do I need to do??):  “Powerful piece by @lizmorrish in THE today; something is going horribly wrong with way academic staff are managed”.

A colleague at yet another Russell Group University which has had its own issues with metrics reflected on the influence of a talk I gave in November 2015: “Thanks too for the mention of our success in [Russell Group University] in resisting a ghastly outcomes-based performance-management system last year. You came and gave an insightful and inspiring critique of it to a UCU branch meeting, which provided us with courage and the intellectual tools to tackle it. I am sure you will continue to play such a role. How can those of us still in universities support you at this time and going forwards?”

After such a show of appreciation, I can honestly say that I do feel supported – anchored, actually – in a community of scholars from which my former employer thought I had been ejected. I am fortunate to be able to continue writing, blogging and reaching out on Twitter and hopefully connecting and influencing that way. So in that sense I am ok. It’s the rest of you left behind that I worry about, so let me make some suggestions.

The people who need our concerted support are those whose academic freedom is compromised because their contracts are temporary or zero hours forcing them into the hire and fire economy of contingent labour. They dread questioning authority and have no real autonomy either in the classroom or outside of it. Thankfully, our union is campaigning on this issue. We must put pressure on universities to take measures for sustainable careers post-PhD.  The University of Birmingham is making a start with its research fellowship scheme – 5 years of research followed by a lectureship. This is a positive development.

We must talk to colleagues and students about the effects that work-related stress is having. When I was still working at a university, I was often sought out by colleagues for these conversations because my research and stance offered reassurance that it was the system and structures which were the problem, not the individual. This remains the case, despite all the wellbeing workshops and employee assistance programs being implemented across the sector. Unless we challenge management-by-metrics, academics will continue to get ill. This recent article in the Guardian Academics Anonymous addresses the embargo on talking about stress and mental health in universities. As my Times Higher piece reveals – there can be penalties for breaking the code of omerta, but we must.

We must resist collapsing our academic identities into a set of data points and spurious proxy metrics for ‘performance’. Let’s not talk about being REFable, or incorporate our h-index into our email signature. Instead, resolve to have conversations about interesting research, and how we add to it or want to integrate it into our teaching.

We can put pressure on our institutional managers to sign up to DORA: the San Fransisco Declaration on Research Assessment. Imperial College has just become a signatory, and several UK research universities have now committed to this pledge not to use journal-based metrics in hiring and promotion decisions.

Universities are held in thrall by the REF, and will presumably be so again with the TEF as long as government uses these mechanisms to control funding. It is sometimes hard to differentiate unavoidable external constraints from gratuitous control of academics’ behaviour. But if we empty academic careers of autonomy, then we risk being left with universities full of dressage ponies.  Let’s resolve to use our own judgement in our ‘self-directed research and scholarship’. We owe that to our students and our disciplines.

We should reject the damaging discourse of ‘excellence’ that has invaded every corner of universities. This is critiqued in an excellent paper by Moore et al. (2017). As the authors point out, excellence is not a discoverable quality. It is, of course, a fiction. At best it is a discursive strategy to normalise the achievements of the most talented and ambitious academics and make everyone else seem deficient by comparison. At worst it is a smokescreen for what Joyce Canaan calls ‘a culture of crappiness’. Moore et al. recommend that we retrain ourselves to evaluate our academic endeavours in terms of soundness and capacity.

And lastly we must ask our union branches to monitor any rise in disciplinary actions against colleagues, and scrutinize the effects on academic freedom, or rise in fear of inappropriate reprisals. There is a perception that there has been more frequent recourse to these procedures, but we need evidence and consistent monitoring.

I am grateful to Agnes Bosanquet who blogs at The Slow Academic. She writes about small targeted acts of resistance (STARS). In a citation she gives these examples: “Individuals were deliberately maintaining their research interests in defiance of perceived [audit]-rewarded tends; departments were actively pursuing collegial rather than competitive practices.”

These are all things we can do individually and collectively to resist the erasure of our academic autonomy by audit and the limits that discourse sets for our sense of achievement.

Some of my colleagues asked me why, unlike Marina Warner in this hard-hitting piece and another,  I refrained from naming the institution I left. The first reason is because my critique has never been intended to single out one institution – the problems are quite manifestly sector-wide. The second reason is more complex. The managers who chose to pursue me with disciplinary action will recognise themselves in the piece. The postmodernists among us would call this interpellation, and in queer theory, individuals are interpellated by shame if they respond to a hailing. They are hardly likely to step forward and claim their ignominy by objecting, in the same way they shut me down the last time. On that occasion they isolated me with a bond of silence. Now I have turned the tables and gagged them. One small targeted act of resistance.

Ten Myths and a Truth from the TEF: Reading the White Paper

Although the Higher Education and Research Bill is still going through parliamentary scrutiny, the Teaching Excellence Framework is about to be implemented and yet we do not know for certain what its effects will be, or even which institutions will enter into it. On the 2nd of December 2016, the same day as students at Warwick University went into occupation against the TEF , the chair of the TEF, Professor Chris Husbands,  published a blog piece entitled Busting five common myths about the TEF. A welcome addition to the critique, I thought, but I felt as though we were reading different documents.  I have been working on Chapter 2 of the White Paper (TEF) and so I checked some of Jo Johnson’s claims against evidence from some of the other publications I have been reading recently. Concealed within the pages of Jo Johnson’s White Paper, Success as a Knowledge Economy, May 2016,  are quite a few contested propositions and ten more myths which Chris Husbands has overlooked.

We hear much of how political discourse operates in a post-truth culture, but one of the key strategies of persuasion is via presupposition – an statement whose truth is assumed without substantiation. Another trick is to make syntactic linkages between concepts which then acquire the appearance of logical relationship. We find both of these demonstrated in the White Paper.

Below I outline myths (quotations and presuppositions from the White Paper) and responses based on evidence and reason.

Myth 1: There is a problem with ‘lamentable’ teaching quality in universities.

Response: There is no evidence presented to sustain the claim. Use of an inflammatory adjective installs the presupposition.

Myth 2: Students cannot make informed choices….These decisions are significant factors in determining a student’s future life and career success, so it is crucial that they represent sound investments. We need to make sure that students have access to the best possible information to make choices about what they study, and the benefits that they can expect to gain from those choices.

Response: Students have a lot of choice of courses, and they make up their own minds by consulting websites, alternative prospectuses, going to open days. There is even metricised data from Unistats  (comparison site which evaluates NSS scores, employment data and graduate salaries – exactly the innovation Jo Johnson thinks the TEF will deliver) and from league tables.

Nouns like ‘investment’ can also operate as presuppositions as the concept is assumed to be inevitable and universal.  ‘Investment’ is presented in crudely financialised terms as ‘return on investment’ or ROI, which presupposes that students are primarily concerned about future earnings. No evidence is presented to substantiate this, even in the face of students continuing to apply for courses where relatively low salaries are likely upon graduation e.g. nursing, creative arts, education, agriculture. We note that ‘investment’ is a polysemic (multi-meaning) term used to reference the expending of economic capital, and emotional/ intellectual capital by the individual.

Myth 3: Robust, comparable information about the quality of teaching – and the components that contribute to it – is not currently available… That is why this Government will introduce the TEF and for the first time bring sector-wide rigour to the assessment of teaching excellence.

Response: A repetition of the presupposition that students do not already have access to this information. As stated above, it clearly is available. If it is not, why have we been pouring money into QAA, institutional reviews,  Hefce, etc. for all these years, if it has not had the effect of ensuring the quality and reputation of the sector? This architecture of quality assurance, though imperfect, has ensured that the UK is one of the most highly regulated and inspected sectors in the world.

Myth 4: The consumer organisation Which? has found that three in ten students think that the academic experience of higher education is poor value, and the issues raised by students in that research included the amount, and quality, of teaching they received, and the extent to which they are academically challenged.

Response: It is good to see a rare appeal to evidence, but perhaps the wrong conclusions are being drawn by the Which? study. This study by Steven Jones, Steven Courtney and Ruth McGinity proposes another interpretation: “Large fee increases mean that university is bound to be seen as exploitatively expensive by students. This does not mean they are dissatisfied with their courses or teaching quality”. In fact, the NSS scores nationally indicate that students are satisfied with their university experience. Can Jo Johnson make NSS a key metric, and then discount it, all in the same policy document?

Myth 5: Clear priorities of students while at university included: “having more hours of teaching”, “reducing the size of teaching groups” and “better training for lecturers”, but there is little information for prospective students on this in advance.

Response: As this study finds, effective student learning does not always emerge from ‘more contact hours’; in fact independent study is more valuable.   Learning may be the first casualty of a popularity-led evaluation like the NSS/ TEF.

Myth 6: Employers report a growing mismatch between the skills they need and the skills that graduates offer.

Response: A study reported in the Times Higher in 2015 shows that universities are doing a good job in developing the kind of skills which employers find useful and “UK employers are still among the most satisfied with their nation’s higher education system (giving it 7.3 out of 10, compared with a global average of 6.8).”

Myth 7: We need to ensure that our higher education system continues to provide the best possible outcomes. These come from informed choice and competition.

Response: This is a logical non-sequitur, but allows a lazy conflation of several unrelated concepts and assumes causality between them. The White Paper assumes that outcomes = return on investment = graduate salaries, and that these will be consequent upon informed choice and competition. Quality of courses, and choice for students, is more likely to emerge from imaginative cooperation between institutions. This would be an innovation worth pursuing.

This study by David Morris of Wonkhe analyses the government’s Longitudinal Earnings Outcome (LEO) data. There are a number of departures from the outcomes-require-competition myth. Prior attainment, i.e. A Level performance, makes a huge difference to graduate earnings, regardless of subject studied.  This raises a question about ‘learning gain’ – also a concern of the White Paper. I’m sure this will present itself as another cudgel to beat less-favoured universities with. However, Morris’ study also identifies a gender gap and a race gap for earnings, which is far less consonant with a learning gain/ value-added analysis.

Myth 8: By removing student number controls and making it easier for new providers to enter, we will create the conditions that will allow choice and competition to flourish. But what is also needed is the information to allow students to determine where the best teaching can be found.

Response: The answer to quality enhancement, we are expected to believe, is the entry of new providers in order to create ‘competition’. Except the new providers will not be expected to fulfil all the expectations that publically-funded universities are expected to address. As this article makes clear, as new private providers have emerged in strength in South America, especially Argentina and Chile, they have not been engaged in research. This, argues the author – Carolina Guzmán-Valenzuela, restricts the number of qualified PhDs who are able to take the higher education system forward.

Myth 9: The Government believes that excellent teaching can occur in many different forms, in a wide variety of institutions, and it is not the intention of the TEF to constrain or prescribe the form that excellence must take. What we expect though, is that excellent teaching, whatever its form, delivers excellent outcomes.

Response: The TEF will have criteria, and metrics, so how can the White Paper say that the form of excellence will not be prescribed or constrained. In fact, that is exactly what will happen as institutions align their priorities precisely to those criteria – which as the statement makes clear, are in any case based on the proxy ‘outcomes’ of NSS scores, retention and most importantly graduate salaries which are high enough to pay back all the money the government has lost in its ill-advised restructuring of HE finance towards what are, in effect, individual student vouchers.

Myth 10: Perhaps the biggest myth of all – as Jones, Courtney and McGinity point out, is Johnson’s claim that the TEF will strengthen the position of students.  It will not – and indeed, the NUS has voted to disengage from TEF. Evidence shows that co-opting students as consumers is damaging to educational experience.

A truth – a veritable truth: There is of course more to university than financial gain, but the idea that excellent teaching occurs in a vacuum, independent of its impact on students’ future life chances, is not one we can or should accept.

Response:  There is a nice hat tip to other justifications of HE, but immediately we see the counter-narrative remains in place with co-reference of outcomes with financial gain, disguised as ‘life chances’. The presupposition is that the most significant outcome of higher education is employment, but as this study shows, economists have often found that education has benefits for society beyond those of the individual – for example in terms of volunteering, social trust, better citizenship (lower crime).

 

Whatever does ail the higher education sector in the UK, the TEF spreadsheet will not fix it. Much more likely is that the government will recruit ‘consumer choice’ as a disciplinary tool, overlooking the needs of scholarship, local economies or student interests, and possibly serving as licence for university closure. By allowing this false reasoning to go unopposed, we risk losing quality, opportunity and reputation within the sector. Here is a link to the Convention for Higher Education website which has some key resources for opposing the TEF and the Higher Education and Research Bill. Organise, and support students in their refusal to co-operate with the TEF and NSS as long as it threatens to raise their fees, waste millions of pounds of their ‘investment’, threaten the reputation of their courses and distort the priorities of universities away from good teaching and research.

 

A Stern Talking To ?

The Stern Review has appeared and HE wonks are rushing back to the laptops they had just powered down for the briefest of summer holidays. So here are my thoughts, based on a skim reading of the report, and also the helpful comments on Twitter from the following: Emilie Whitaker @Dr_EmWhitaker , Steven Jones @StevenJones_MCR David Wright @WrightDW  Daniel Grey @djrgrey  Mike Ratcliffe @mike_rat and James Wilsdon @jameswilsdon.

It is clear that Stern endorses the continuation of the REF. This is no mutinous denunciation of research assessment. But then that was never in the terms of reference with its forecast of ‘future iterations of the REF’. He has, though, proposed fixes to escalating costs, institutional gaming and some of the impediments to providing a true picture of UKHE research, such as requiring a demanding output-impact case study ratio and ‘selectivity’ of individuals submitted for assessment. There is also a welcome focus on support for interdisciplinarity which many have felt was overlooked in REF2014. Derek Sayer’s important criticism that the REF is a parochial and navel-gazing exercise is addressed; at last, international researchers are to be invited onto panels (Para 109).

One of the stated imperatives of Stern is to reduce the workload of the exercise. He recommends a strong reduction in the average number of ‘outputs’ submitted per faculty member (Para 69) which at this point is looking like a baseline of 2, or even fewer. There will still be sampling of outputs by unit of assessment (Para 71). One more priority is to reduce gaming of the system, and to this end Stern has recommendations about who owns ‘outputs’, and an end to ‘selectivity’.

The most radical of the recommendations in terms of its implications for current practices is that there should be no portability of research ‘outputs’. Stern feels that they should remain returnable by the HEI which ‘helped to produce’ them. This raises the issue of who owns research ‘outputs’ and is guaranteed to cause indignation among the more mobile academics, and also Early Career Researchers who may have had a series of short-term contracts at different universities. It would, however, reduce the likelihood that avaricious institutions would poach individual researchers if they were entitled only to ‘outputs by the individual that have been accepted for publication after joining the institution’ (Para 74). There is no doubt that this will hit the perceived ‘marketability’ of individual academics, but perhaps Stern has rather harshly assumed that the only reason academics leave universities is because of their own ‘rent-seeking behaviour’.

Recommendation 1 is that all research-active staff should be returned in the REF, noting that ‘exclusion and the associated stigma are being driven by factors that are not wholly related to the quality of an individual’s research contributions and potential’ (Para 64). While this is commendably inclusive, and recognises that excellence in research may be found anywhere, it might also act as an incentive for universities to reassign staff to career-limiting teaching-only contracts. This fear is mitigated by the recognition that the proposed TEF looks favourably on research-led teaching (Para 112). Like other high-level communications and policy documents, the Stern Review also presupposes the implementation of the TEF, even though (in July 2016) this proposal is still being debated in parliament.

There seems to be some sanity at last on the notion of research impact. This should be based more generally on research activity, and not dependent on particular ‘outputs’. Also, assessment of impact need not be confined to socio-economic impact as it was for REF2014. Stern recommends that it ‘should also include impact on government policy, on public engagement and understanding, on cultural life, on academic impacts outside the field, and impacts on teaching’ (Recommendation 7).

From my reading, I felt my antenna twitch at a couple of potential Trojan Horses. Stern allows scope for the use of metrics, while cautiously invoking James Wilsdon’s ‘Metric Tide’ report of a year ago. Once metrics are allowed to stand as proxies for the health of a unit, then inevitably this lends permission for greater infiltration.

Indeed, the primary way the research environment will be assessed will be via metrics (Para 48). It seems perverse not to allow for a qualitative statement in this particular area. One wonders how an environmental template based on metrics will facilitate the ambition to capture ‘the contribution that its academics make to the wider academy (‘academic citizenship’)’ (Para 88). Recommendation 4 states that panels should continue to assess on the basis of peer review but may supplement this with metrics, as long as they are transparent about their use (Para 105).

Sadly, there is no recommendation to commit institutions to become signatories of the San Fransisco Declaration on Research Assessment. HEFCE is currently a signatory, although in a rather insouciant way. Instead, Stern invokes an aspiration to standardise the metrics supplied to various bodies whose incompatibility currently stands in the way of more widespread use. There seems to be an invitation to UKRI to become the overlord in this integration of research metrics (Para 107). Will they, too, sign DORA?

The REF needed a stern review, and it has got it. It offers enough clarity to subdue even the most feverish of vice-chancellors for a weekend. It seems to be informed by principles of parsimony and lean management in contrast to the speculative profligacy of the White Paper’s TEF. If we must live with the REF, an ‘iteration’ whose more rapacious and distorting inclinations have been curtailed is preferable.