How not to measure teaching quality and learning gain

I blogged previously about learning gain just after the General Election.

At that point there was not much to go on but a hazy promise in the Conservative manifesto to “introduce a framework to recognise universities offering the highest teaching quality… and require more data to be openly available to potential students so that they can make decisions informed by the career paths of past graduates”.  On July 1st 2015, Jo Johnson added some more details; the teaching REF was intended to be outcomes focussed, in other words, not focussed on the kinds of ‘quality’ processes that universities had perfected over the last 20 years. The Conservative government hoped to devise a test of learning outcomes that, anticipating the skills of the best schemers in universities, would not be open to gaming.

In the Times Higher on 23rd July 2015, Julia King, Vice-Chancellor of Aston University, made this comment about the Teaching Excellence Framework (TEF):

First, it needs to measure the right things. It cannot be a superficial extension of the data provided through the Key Information Set, a variant on the Quality Assurance Agency’s higher education review or some rehash of the subject league tables that drive universities to offer higher and higher proportions of firsts and 2:1s.

Too right, and while I don’t agree with all her conclusions, I appreciate her drive to ensure that the TEF is based on “a properly evidenced measure of that quality”.

It was disappointing then, in the same week, to read a guest blog hosted on Wonkhe from Edward Peck, Vice-Chancellor of Nottingham Trent University. The piece started with a critique of the validity of current DLHE (Destinations of Leavers from Higher Education) data, and I became hopeful of a rebuttal of some of the cruder measures of graduate success. However, in the piece, entitled Finding New Ways to Measure Graduate Success, Peck explicitly rejects measures which, while not the sole determinants of teaching quality, might at least impact positively on the student experience. On the list are Staff Student Ratios and spend per student, which he regards as perverse, and rewarding inefficiency. As I read on, disenchantment grew:

“Secondly, the forthcoming availability of HMRC tax data to HESA and the Student Loans Company means that we could use a robust measure where we can select the census point at which we present data on average earnings by university and/or by course. This would not be dissimilar to the approach some rankings take to MBA programmes. With secondary education performance data also being brought into the mix, we have the hope of finding a much needed way to measure added value or learning gain”.

Now, I’m all in favour of looking at learning gain and allowing that to inform improvements to the student experience. I’m not in favour of releasing irrelevant and crude data in a league table. I’m still pondering the author’s logic when he presents graduate salary levels as an indicator of ‘value added’ by universities, and appears to see this as a proxy measure for learning gain. Universities are repositories and generators of research and knowledge, not factories for ‘salary men’. I would have hoped the nation’s vice-chancellors would have challenged those assumptions, not propounded them.

Here are five reasons why the equation of graduate salary, teaching quality and learning gain is unfounded.

  1. The glass floor effect. On July 26th The BBC news lead story was that middle class families are able to prevent their children from sliding down the social scale, regardless of talent. Clearly, then, graduate salaries are more likely to correlate with social class.
  1. The continued existence of a gender pay gap underlines how fanciful it is to assume that a high salary results from ‘value added’ higher education. Have only male graduates received better teaching? Data from the Fawcett Society indicates a pay gap of 19.1%. Admittedly, this figure encompasses pay for all workers, not just graduates, but points about the motherhood penalty, and outright discrimination are still valid, even in universities. Talk of perverse incentives! Would universities accept more white, able-bodied, middle class males onto courses, if they were likely to earn higher salaries?
  1. Economies are not stable. Middle class professionals continue to feel the force of austerity caused by bankers’ irresponsibility. ‘Generation rent’ may never attain the standard of living of their parents, but this has resulted from a failure of the generational compact, not the failure of universities.
  1. Similarly, we cannot predict which subject areas may prove lucrative. Recently, graduates of any discipline who work in finance have received higher salaries, while pay and employability for graduates in IT and bioscience have taken a downturn.
  1. The SBEE (Small Business, Enterprise and Employment Act 2015) has unleashed a huge data salad of graduate earnings, student loan repayment and course/ university attended. . At best, there may be an association between your earnings and your alma mater. This raises the question of construct validity – a notion drilled into all educators – make sure you have the appropriate measure in order to draw conclusions. Association of any set of measures does not indicate causality, so Peck’s suggested metrics are not even valid as proxy measures of learning gain.

Those of us who care about the future of higher education in the UK cannot let these lazy assumptions dominate the agenda of academic institutions. The data points may link up, courtesy of the SBEE, but the logic does not. It would be ironic if we allowed universities, of all places – interrogators of assumptions, busters of myths and challengers of fallacies – to be led by spurious metrics for what will probably turn out to be immeasurable.

teaching quality venn


The paradox of the ‘under-performing professor’

This post has been inspired by an apparent declaration of hostilities towards professors in a number of universities. The weapon of choice has been performance management, and some aspects of audit culture have been liberated from their usual role of absorbing academics’ time to becoming instruments of punishment.

In universities we have seen a deprofessionalization of academic staff which has manifest itself in a number of ways. In, many areas, disciplinary groups have been broken up and atomized across the university, in response to management‘s fear of ‘silos’. In others, productive interdisciplinary groups have been disrupted by reorganizations which have obstructed innovation. In at least one famous case, an entire centre was vandalized (Centre for Contemporary Cultural Studies at the University of Birmingham in 2002) because its members refused to surrender to the neoliberal commandment that research must be a competitive and self-important process.

The one thing that academics were permitted to retain was a system of academic esteem bestowed by promotion to professor. This was trusted to reward talent, reputation and diligence, but in recent years, even those who attain professorial rank are subject to this regime of never quite ‘becoming’. In several UK universities, the intrusive gaze of Human Resources has recently fallen on alleged ‘under-performing professors’. I have managed to obtain performance criteria documents from a number of universities where professorial targets have been revised.

In any sane university, to talk of ‘under-performing professors’ as a generic description, would be recognized as pure incongruity; since Human Resources decide the ever-ascending criteria for promotion to this level, they might be trusted to not betray their own judgment. There seems to be some degree of ‘moral panic’ among senior management teams as in many universities, crude targets for grant income are now being set for individual researchers. Increasingly in universities, as well as undergoing six-monthly performance reviews (as frequently as newly appointed probationers), professors must now meet exacting criteria for ‘quality’ of publications. Progression to the next professorial level must be achieved within five years, and this depends on meeting certain ‘drivers’, which include securing a research grant as PI every two years, producing REF 3* and 4* ‘outputs’, supervising graduate students, producing a significant impact case study, leading high-prestige international collaborations, and of course, continuing to teach. Failure to meet these expectations will result in the public humiliation of improving performance procedures, and possible demotion. No accrual of reputation can be permitted; the criteria must be met every year, not just over the course of a distinguished career. In this way, any prestige associated with the rank of professor must be considered temporary, as is its tenure. Professors, then, have been made to join the expanding precariat of the academy. Ben Knights (2013) cites Sennett (1998), who recognizes that “a regime which instills insecurity, in which you are… ‘always starting over’ is inimical to the longer term processes of memory and imagination.

This is the society of control outlined by Deleuze (1990).  Foucauldian (sequential) disciplinary regimes (Morrish, 2011)  give way to ones in which, just as one hurdle is surmounted, another, higher one presents itself, with the end point always at the far horizon. We find this reflected in management documents on performance review with a lexicon of journeys, milestones and checkpoints, but the individual is never allowed to arrive at the promised reward. Gatekeeping measures such as the imposition of perpetual training, perpetual review of publications or multiple-staged applications for promotion, must be endured, even to participate.

Performance management has recently been under scrutiny by the press, academics and their trade union, University and College Union. The death of Stefan Grimm in September 2014 shocked the academic community. A very moving appeal to the academic community appeared on this blog:

Professor Grimm held the Chair in Toxicology at Imperial College, London, and he took his own life after being threatened with performance management procedures when he was deemed not to have brought in ‘prestigious’ grant money.

His obituary on the Imperial College website reads:

Over the past 20 years, his work to this scientific field includes 50 publications in top-ranked journals, two books, more than 3000 citations and 5 patents on innovative strategies for screening novel genes involved in cell death pathways and new anti-cancer genes. Professor Stefan Grimm chaired and co-organized international conferences and served as reviewer for research-funding organizations and many international scientific journals. Recently, Stefan was elected as fellow of the Society of Biology. (

This hardly looks like the profile of an ‘underperforming’ professor. His crime, though, was that he prioritized science rather than the accumulation of capital. It took seven months for Professor Alice Gast, the President of Imperial College, to make a public statement on Stefan Grimm’s death. In an interview on the BBC Radio 4 Today Programme, on 17th April 2015, when asked directly about the case, she offered this oblique comment:

Professors are under pressures. They have a lot on their plates. Professors are really like small business owners. They have their own teaching to perform. They have their own research and they have their research funding to look after. They work with teams of post-docs and post graduate students. Then some of them work on translational work and develop entrepreneurial and new companies and spin outs. It’s a very highly competitive world out there. The collaborative nature and the way in which we’re moving towards highly collaborative work I think helps because one starts to recognise that you can’t do it all alone. You need a team. You build a team with the very best colleagues. You have not only that interplay between the different backgrounds and disciplines but you get the new ideas that are generated by bringing diverse people together. (

There has been a shockingly rapid move from entrepreneurship as metaphor, to a state in which it is both literal and mandatory. It features as a ‘key competency’ in academic job descriptions, and there is now an expectation that professors will earn their own salaries and research expenses. In addition, professors are seen as a kind of Praetorian Guard who will build a university’s brand with ‘outreach’ activities such as media interviews. As we know, institutional branding is about the manipulation of appearances, but when that is made a priority in a professor’s workload, you know that academic values have been forsaken. It is a world which creates posts like an Associate Dean of Eureka Moments (Bristol University 2015) and a Pro Vice Chancellor of Ambition Innovation and Student Satisfaction (Anglia Ruskin 2015).

It is common in the performance management documents I have collected, for reference to be made to ‘stretching objectives’ which are purported to sit in between an individual’s ‘comfort zone’ and the  ‘panic zone’. ‘Stretching objectives’ are presented as desirable, but objectives which place individuals in their comfort or panic zones are not. There is a disturbing presupposition in this discourse of comfort zones. To be asked to go beyond it makes the patronizing assumption that one’s life is normally comfortable. It certainly reveals that those charged with auditing and defining these comfort zones are fortunate in this way. It is a discourse which permits no acknowledgment that the employee may find teaching or research extremely stressful, at least some of the time. Their domestic circumstances may add additional stress – illness of a child, the loss of a partner’s job, death of a parent – these may all lower the threshold of discomfort at work. The managerial class, who can at least assuage some of their discomfort with a larger salary, should check their privilege and ‘think outside the box’ they have just casually ticked.

It is not clear what results university managers expect to emerge from a system of unattainable targets, constant surveillance and audit, and the knowledge that any dip in ‘performance’ may see their contracts terminated, but the death of Stefan Grimm should have brought this kind of disciplinary regime to a swift halt in any ethical institution. In some universities, professors are subject to an inversion of operant conditioning whose ‘incentives’ would be recognized by Milgram, not Skinner. In all this talk of drivers, stretching, and comfort zones, did anyone stop to think of the psychological risk of treating professors as though they were computer processors with a limited life and inevitable disposability? I am not a professor, but many of my friends are. They are all passionate, creative, rewarding colleagues and professionals. They are remapping their fields for others to follow. The fact that they may not be one of the 15 in 100 who wins a research grant is really no reflection on the significance of their work.  They are people whose primary identity is defined by their scholarship. Did nobody in HR raise an objection that treating a professor like this is inhumane, because it certainly makes me weep? The last word on this belongs to Stefan Grimm. “They treat us like shit”, he said at the end of his last email to colleagues. And then he ended his life.


On September 7th-8th 2015 a seminar on Language of Money and Debt was held at Roehampton University. The organizers were Dr Annabelle Mooney and Dr Evi Sifaki and I was a keynote speaker. This event fell close to the anniversary of Stefan Grimm’s death. We felt it should never be forgotten. In response to the Music for Deckchairs blog piece, which asked us all to do the academic equivalent of ‘putting our bats out’ for Stefan, we decided to build a cairn of books. Each participant was asked to bring a book to share with our students, our colleagues and ourselves, which might help us to deal with these pressures. It is the most fitting memorial we could think of.

Stefan Grimm cairn


Deleuze, G. 1990. Postscript on the Societies of Control. Cambridge, MA: MIT Press. 3-7.

Ben Knight    Knights, B. 2013. ‘Politics and enhancement: the English Subject Centre’ in (eds.) Deborah Philips and Katy Shaw. Literary Politics: The Politics of Literature and the Literature of Politics. Basingstoke: Palgrave. pp181 – 193

Moriarty, Philip. 2015.  The use of raw grant income as a performance target has got to go – now. Times Higher. June 18th.

Morrish, L. 2011. Con-Dem-Nation and the attack on academic cultures. Campaign for the Public University. November 2nd 2011.

Gender and performance in the neoliberal academy

In my previous blog post, for ‘A feminist leaves NeoLiberal U’ I was responding to some of the concerns that a colleague had raised, in her letter of resignation, about her gendered experience of the neoliberal, competitive, speeded up academy. My colleague is the mother of young children, and has found the multiple demands of teaching, research and administration overwhelming, especially in a context of institutional panic about the REF and league tables, and escalating expectations of ‘excellence’. Below I offer more thoughts on the particular impact of the culture of restlessness on women in universities, as this seems to have touched a nerve among readers on the blog and on Twitter.

In the UK, performance review and REF submission loom large as ‘drivers’ of academic anxieties. In the US, tenure and promotion take their toll, especially on women, who may face domestic as well as professional expectations. This is a reflection written by a female US academic about her upcoming mid-tenure review:

For some of us, it’s not that we are afraid to lean in. It’s that we have jumped in head first and are barely treading water even when we are considered “successful.” It’s not that my success has come at the expense of family or that my career advancement has been stifled by raising a family. It’s that my success in academe is simply not the kind of success that I envisioned myself. Success should feel good, make you beam with pride, feel as if all your hard work was worthy of something bigger. I envisioned, and frankly deserve, a type of success in which the next panic attack isn’t just around the corner and in which supportive spouses don’t feel like they must resort to ultimatums to cultivate a meaningful family life. (Sangaramoorthy, 2015)

As this example shows, it is important to recognize the differently gendered effects of the neoliberal preoccupations with competitiveness, efficiency and increasing productivity. Without wanting to appear essentialist about the particularities of the effects, we need to take into account the realities of many women’s lives. Lynch (2010) and Evans (2010) both refer to the ‘careless’ university which only rewards ‘careless’ employees. It is your bad luck if you have caring responsibilities which limit the time you can devote to ‘productive’ work.  Shame on you if you wish to mentor a younger colleague, and overlook a publication deadline. Capability procedures for you, if you happen to lose the lottery of research grant ‘capture’. Women, writes Evans, must be prepared to perform according to the metrics of success that have been derived according to norms of masculine lives.

So far, I haven’t even addressed the extent of the extra imposition faced by women of color. As well as acting as role models and sources of counselling and affirmation for students of color, institutions often burden such faculty members with promoting and building diversity. These assignments soak up time, and expectations of published outputs are rarely adjusted accordingly.

And these are the good institutions that claim to care about diversity. For the most part, it is barely acknowledged except in the form of an Equality and Diversity department, there to ensure legal compliance and statistical monitoring. There is no intervention against the compulsory conformity, the inscriptions applied by racist structures on bodies seen as different – these all belie institutional claims and commitments to diversity. I have borne witness to the exclusion of several black women colleagues: their careers casually thwarted by neglect; their unique contribution to the student body forestalled.

For all of us there is a common theme. Our professional lives are dominated by the need to provide discursive evidence that we are compliant with the managerial regime in the form of performance management reviews, teaching evaluations, student satisfactions surveys, research excellence frameworks. Failure to enter into the discourse results in illocutionary silencing, since one has become literally unintelligible to the managerial mind.  By locating critique outside the range of the sayable, our resistance is blunted (Davies and Bendix-Petersen, 2005: 85). The discourse of audit, as Strathern (2000) explains, is often about ‘helping’ people to monitor themselves, and indeed, Gay Tuchmann (2009) has said that we do this as reflexively as a diabetic pricks her finger.

What I found so rare in my colleague’s letter of resignation is that, even at a point of desperation, she has somehow found the reserves of self-worth to think her way outside of this. There is a peculiar force field to audit culture and the rituals of verification (Power 1999) that go with it. Regimes of performance management formalize these to the extent that our whole academic identity has been re-shaped by a series of managerially-imposed criteria, which for many of us, are simply incongruous with academic values and aspirations.

The response to my blog post on Twitter was heartwarming. I feel encouraged. I have no idea how my colleague is feeling, but I’m guessing there is a poignant sense of a supportive community, though intangible, invisible and located somehow out of reach. This is the clandestine academy that Thomas Docherty has written about. There is an urgency, as I will argue in my next piece, for making our views known to management – to resist the discourse one performance indicator, driver and dashboard at a time.


Davies, B. & Bendix Peterson, E. 2005. Neo-liberal discourse in the academy: The forestalling of (collective) resistance. Learning and Teaching in the Social Sciences 2 (2): 77-98.

Docherty, T. 2011. The unseen academy. Times Higher. 10th November 2011.

Evans, M. 2010. Coercion and consensus in higher education. Learning and Teaching in the Social Sciences 3 (2): 39-54.

Lynch, K. 2010. Neoliberalism and marketization: the implications for higher education. European Educational Research Journal 5 (1): 1-17.

Misra, J. and Lundquist, J. 2015. Diversity and the ivory ceiling. Inside Higher Ed, June 26th.  Accessed 3rd July 2015.

Sangaramoorthy, T. 2015. A hockey mom seeks tenure. Chronicle of Higher Education.

Strathern, M. (ed) 2000. Audit Cultures: Anthropological Studies in Accountability, Ethics and the Academy. London: Routledge.

Tuchman, G. 2009. Wannabe U: Inside the Corporate University. Chicago: University of Chicago Press.

A feminist leaves NeoLiberal U

Liz Morrish replies to a feminist colleague’s letter of resignation. 

I was very sorry to read your letter of resignation. I was, though, delighted that you decided to circulate it among colleagues at NeoLiberal U, along with an article, rapidly becoming a classic, if my Twitter feed is any predictor, by Mountz et al in the Great Lakes Feminist Geography Collective, offering a manifesto for a slower pace of academic life. This is what you have not found at NLU, and you weren’t prepared to go on sacrificing the possibility of intellectual creativity, family life and personal space forever. Sometimes principles have to be lived by, because that’s the right thing to do. NLU doesn’t seem to have any other principle than to ‘maximize the staffing resource and leverage the maximum from the academic contract’ (I paraphrase).

It has been a long time since we sat down and discussed all this. That is just your point, though. In the speeded up university, with its distorted constructions of academic ‘productivity’, schedules are crafted to eliminate the necessary practices of caretaking. In my field of work, this is known as ‘relational practice’, and in its most benign form, it is attributed to women. I haven’t been doing much relational practice recently, and have been contemplating this neglect during a period of sabbatical. There is a tendency at work to hole up in offices, and scurry past colleagues you know to be in need of support. It is emails like yours that make me aware of how many of us inhabit the same private hell of alienation, shame, stress and guilt.

All of this is well documented in the Mountz piece, and by Ros Gill, Maggie O’Neill, Mary Evans, Kathleen Lynch, Bronwyn Davies and Eva Bendix-Petersen, Priyamvada Gopal and many others. The critiques of audit culture are mounting, but not as fast as the university ratchets up its demands for ‘accountability’. You found it impossible to prepare and teach new modules each year, do good research, cope with constant change and restructuring, and still be told that you are not working hard enough. I agree, it is uncongenial and it is abusive. As new ‘benchmarks’ for 3* and 4* ‘outputs’ are set by managers who seem oblivious to the demands of our profession, you looked ahead and found the future unsustainable. Sadly, you are not the first, or the last. We have let many hugely talented, capable, and caring, women slip out of the academy without an attempt to address the issues which prompt their departures. It is truly a care-less institution. There is official denial that there is a problem with staff retention, and to frame it in HR terms, your significance must be set against the importance of ‘the role’. There is only ‘the role’; you were merely the temporary place holder. And as you point out, there will soon be some newly-minted PhD who will be prepared to work the 60-70 hours a week necessary to fulfill ‘the role’. In the end, you had the strength and integrity to realise that academic freedom cannot survive in the hot-house culture of perpetual surveillance and ‘kaisan’ that has become the way of life at NLU. What you have observed – a system that stifles intellectual endeavor as much as it considers itself productive and dynamic – is termed ‘acanemia’. It disguises its damage by pretending that its individual parts are malfunctioning. You have correctly diagnosed the system as being sick. This has not prevented you from being harmed however, by what Ros Gill has called the ‘hidden injuries’ of higher education.

I wonder what was the last straw? The broken promises on workload equity and research time? The shifting goal posts for promotion? The time wasted on pointless change? The lack of collegiality? The authoritarianism which ruptured the working relationships you had developed? Or was it the growing awareness that immediate managers have as little power to improve your conditions as you do?

Given the totalizing reach of the in-corporat-ion, I am pleased to see you refusing to allow yourself to be judged by those absurd binaries and exclusions which govern our working environment. The ‘performance reviews’ which consider only the visible and measurable workstreams. And the monetizable, of course. Research active status will now be a designation earned only by a new, homogenous and care-less elite. It will not be allowed for the slow, careful writer who wishes to do scrupulous and yes, pleasurable work. In this obscene inversion of academic values, such a person may well be subject to the discipline of ‘capability procedures’ if we read the strategic plan. For capability now equals not scrupulosity of process, but speed and quantity of ‘outputs’, possibly augmented by the requisite citations and H-index. This is a world of multiple Neoliberal Us which have decided to eat their young. We have lost a good dozen in the last few years. Those former colleagues are now beyond the reach of REF, NSS, QAA, PDR and a whole host of officious, soul-sapping bureaucracy. Now that you are leaving, those others among us who critique the regimes of performance management, surveillance and ‘output measures’ are a little more isolated. It is affirming to hear similar perspectives from younger voices which discount managerial narratives of entrenched positions quarantined among peevish golden agers.

I’m sure, given the opportunity you deserve, that you would have ascended the ladder and made it all the way to professor. But, this accolade, too, has been reformulated to reflect an institution which never reaches its goals. No process is ever allowed to reach fulfillment, and so it is with academic careers. You thought you had arrived? Think again! The collective vice-chancellors of the UK seem determined to declare the area a professor-free zone, as one after the other, universities have declared war on their professors Taking your bow after an inaugural lecture is your first step into a new precariat. It can only be a matter of time before HR departments are instructed to replicate that strategic insecurity among the rank and file academic staff.

So get out and stay out of academia would be my advice. It is unlikely to get better. Three years from now, big data will see us all reduced to just a blinking light on a faculty ‘dashboard’. But I would like to thank you for your work. I’d like to thank you for not just turning your back and leaving silently. Thank you for calling out the insanity, and refusing to submit to false definitions and measures. You leave with the admiration of your colleagues, which to me, is the only prize still worth having.

Fictions and Contradictions: Performance Management in Higher Education

Liz Morrish argues that the assumptions of performance management in higher education reside in the world of managerial fictions. It is a process riven with contradictions which require urgent rethinking.

Performance management has become a feature of the higher education landscape in the last decade. This definition is offered by Franco-Santos, Rivera and Bourne (2014), “At work, individuals are said to perform when they are able to achieve the objectives established by management. Organisations are thought to perform (or to be successful) when they satisfy the requirements of their stakeholders and are more effective and efficient than their competitors.” [my italics]

Recently, this document was circulated on Twitter. It is the assessment form required for performance review meetings at a non-aligned post 1992 university.

IMG_20150620_134505 (1)

It is important, as a first step to framing resistance, to recognise the presuppositions and ideologies enshrined in this document. One of the major presuppositions that underlies the use of performance/performing is that there are standard ‘key indicators’ of performance which are invariable. It is presumed that management will decide what these indicators are, and that academic staff performance can be objectively measured using them.

The most striking omission from the checklist above is any recognition of normative assumptions about academic work: that it involves teaching and research; that it is absorbing; that it involves insight, imagination, networking, diligence; that it is rather indefinable in scope, and quite possibly not a good candidate for this type of one-size-fits-all assessment.

The scope of the job, we assume, is contained within the unexpanded categories of Quality of Work; Quantity and Output. The contradiction lies in the under-informing of the designation ‘satisfactory’. In a working environment where assessments are multiple (NSS, REF, QAA, peer teaching reviews etc.), and ever more searching and fine-grained, what are we to take from ‘satisfactory’? It seems to leave open a return to a default ‘unsatisfactory’ if perceived under-performance in one of the measures should become unbundled from the totality. The semantic instability of the evaluative adjective ‘satisfactory’ means that staff will never be ‘performing’ perfectly. It will always be possible to claim that there are ‘areas for improvement’, leaving the apraisee exposed to capricious revisioning. On the issue of Quality, there are also questions of whether a manager’s expertise can always extend to appraising quality of research, particularly when the parameters for exercising judgment are far from clear.

‘Quantity’, similarly, is unrevealing. The presupposition is that more is better, so how does this sit with likely institutional policies on work-life balance and stress management? And what measures are being used? Hours logged in the classroom? Or evenings spent answering students’ questions on email. Or thank-you cards sent by finalists? Are teaching and research both considered ‘outputs’? We note an ironic choice of language in ‘quantity’, especially given the non-judgmental use of ‘engagement’ that universities usually apply to student work.

Job knowledge might be an issue for new academics in post, and so seems superfluous beyond the probation review. Digging a little deeper, though, many of us observe that ‘procedures’ are shifting and transitory under conditions of volatility in higher education regulation. Given the secrecy and lack of consultation with which those changes are imposed, and the general regulatory ‘churn’, to require familiarity, is like asking appraisees to apprehend a mirage.

Perhaps the most pernicious trait desired in this forlorn framework is ‘attitude’. By what ‘benchmark’ are we being evaluated? I would counter that there are occasions where a negative attitude is beneficial to the academic community. Are we required to embrace ineffectual managers, or unenthusiastic students? Should we tolerate abusive phone calls from parents with equanimity? My attitude is my attitude, thanks. I don’t need your evaluation. In any case, how do you suggest remedying ‘attitude’ without seriously compromising my initiative (another category), or academic standards? Never underestimate the positive power of negative thinking, is my motto.

Appearance is yet another superficial and subjective judgement. Here are my prejudices: no sweat pants, no logo t-shirts, no leggings, no flip-flops. But if you ask the students, they are generally very welcoming of lecturers who mirror the informality of their ‘customers’. Possibly, though, this dispensation applies much more to male lecturers than to women. So what is being evaluated here, apart from some manager’s notion of gender-normed corporate dress code conformity?

Attendance and punctuality suggest a rigid and compliant personality, which is undermined by the desired qualities of ‘initiative’ and ‘flexibility’. The latter, particularly, is likely to escape actual validation. Our flexibility makes itself known to family as work bleeds into leisure and domestic life. The continual peeking at email, the agenda planning that invades a run, the feelings of guilt at taking a whole Sunday off during the marking season. It makes itself known as we submit to ridiculous marking turnaround times in order to satisfy intensifying demands for more feedback. It shortens our careers.

This is the world of university performance management at its most unthinking. Cheeringly, it is beginning to be challenged even from within. According to their recent report, the Leadership Foundation for Higher Education would find this appraisal ‘not fit for purpose’. The report  distinguishes between stewardship and agency approaches to performance management, and urges universities to consider a more flexible application of these. Stewardship approaches “focus on long-term outcomes through people’s knowledge and values, autonomy and shared leadership within a high trust environment”. By contrast, “agency approaches focus on short-term results or outputs through greater monitoring and control”. I can probably guess which one seems more familiar to most academics, for whom autonomy, shared leadership and high trust working environments reside in the folklore of a previous generation.

However (to gleefully break a Govian rule), institutions with a mission that is focused on “long-term and highly complex goals, which are difficult or very costly to measure (e.g., research excellence, contribution to society)” are more likely to benefit from incorporating a stewardship approach to performance management. If we keep up the pressure, if we call out the pointless appeasement of what Power has described as ‘rituals of verification’ (1997:1), if we undermine the assumptions, then perhaps managers will start to align complexity of mission with a more delicate implementation of performance management.


Franco-Santos, M., Rivera, P. and Bourne, M. (2014), Performance Management in UK Higher Education Institutions, Leadership Foundation for Higher Education, London.

Power, Michael. 1997. The Audit Society:  Rituals of Verification. Oxford: Oxford University Press.

The Outcome of Limiting Learning

Liz Morrish argues that learning outcomes are really not that important for either lecturers or students.

This post is about creating opportunity by freeing ourselves from some of our learning, teaching and assessment practices which set limits on the development of important graduate abilities like understanding that there are different forms of knowledge, different sources of knowledge, and that all knowledge does not originate from authority. Students need to integrate this awareness into the process of developing confidence, resilience and academic judgement. Indeed, an earlier post on the so-called ‘teaching REF’ argued that this is precisely the sort of learning gain which will be assessed. 

Educationalist Julian Stern (2013) tells us that the most essential characteristic of a successful educational experience is SURPRISE. Over the past ten years in UK higher education, we have been sucked deeper and deeper into the conformity exacted by regulatory bodies, but also of our own design. We are in danger of offering a rather repetitive, initiative-draining experience to students. I offer examples of two overly-restrictive practices: learning outcomes and formative assessment.

If we teach in universities, I’m sure we all do a decent job of drafting module specifications. We all know how to harmonize our objectives with subject and course objectives, cascading from benchmark statements and aligning with our university’s nominated graduate attributes. And then we put them in a folder and forget about them. I have yet to meet an academic who can tell me the learning objectives of any of their modules without consulting the specs. They can, of course, tell me all about the content, and what they hope the students will take from the module. Even transferable skills are more relevant to the learning and teaching process than learning objectives. So let’s free ourselves from that particular constraint.

Assessment has become more like a contract of sale than an educational experience. Anyone who has been in universities for a decade or more will recognise this scenario. You have a curriculum review, and there you discover the previous guidelines have all been reversed. In the last exercise I experienced, we did a quick 180 and overturned the framework which had considered multiple forms of assessment a virtue. No more assessment for learning, or assessment for equality and accessibility. Instead, we should confine ourselves fewer types of assessment, and fewer points of summative assessment. Intermediate feedback should be provided by formative feedback, AKA teaching to the test, and closely aligned with clear indicators of achievement against learning outcomes.

The danger I see in both learning outcomes and formative assessment is that we evacuate any possibility of surprise in learning. If it’s the 5th of November, it must be Semantics. Take the online practice test next week. But more damaging for the prospects of our students is that we have emptied our curriculum of any risk that they will need to exercise their own academic judgement. If I set a project brief for students, they expect me to advise on the research question, the nature of the data, the number of subjects, and the methods of analysis they should employ. This is the nature of formative feedback. I often feel I am assessing my own work. I recognise that students would benefit from pursuing group work, but even then, anxieties about dilution of individual marks compromises that learning experience, and makes it a hazardous venture for both students and staff. Formative feedback requires endless rehearsal for the summative event. Far from reducing anxiety, it makes assessment loom very large all through the module. What if I could assess my students on the basis of what they learn from risk taking, exercising judgement and initiative?

It is a long time ago since I taught in the USA, but I do remember that I had much more opportunity for flexibility of delivery and assessment. I could give them an unannounced quiz anytime I wanted. I could set an essay. I could walk in and announce I was cancelling the final exam. I could give them all As if they deserved them. If a visiting speaker was coming to talk on a relevant topic, I could assign the class to go, and expect a summary paper from each student. There was a syllabus, but I felt free to deviate from it. Alas, this Garden of Eden no longer exists, as the US has led the way in consumerist approaches to higher education.

So what if we had the flexibility to be responsive to current events? I was incredibly lucky one year, that former Australian Prime Minister Julia Gillard gave *that* speech on sexism and misogyny on the very day I was approaching the topic of sexist language in my module. What a gift, and what an inspiration to the class that day. But what if I had been scheduled to teach another topic? Learning opportunity missed. Learning is a wandering, progressive, transformative process. Yes, we need to impose clarity, structure and analysis, but let’s not make that the ultimate goal of all our modules, because we are missing some really important learning opportunities. Perhaps we should assess the strength of modules with just one question on course evaluations: “what surprises have you had in this module?”

Not Doing a Thomas Cook

The phrase “doing a Ratner” has its origin in the famous address given by Gerald Ratner, at the time, Chief Executive of the Ratners Group of jewellers, to the Institute of Directors, when he said this about his company’s product: “We also do cut-glass sherry decanters complete with six glasses on a silver-plated tray that your butler can serve you drinks on, all for £4.95. People say, ‘How can you sell this for such a low price?’ I say, ‘because it’s total crap'”.

Since that incident, companies have sharpened their sensitivities to reputational damage, either deliberate or inadvertent. This concern has only deepened with the growth of social media. In fact, absolutely the quickest way to get a reply from any customer service outlet is to call them out on Twitter.

This week, a sad and complex case came to a close. Thomas Cook travel agents had spent nine years avoiding taking responsibility for what a recent inquest has found to be a breach of their duty of care to a family whose children lost their lives due to a badly-maintained heating appliance in a holiday hotel in Crete in 2006. Finally, in May 2015, the chief executive took responsibility, but only after a large insurance payout to the firm had attracted further negative coverage in the media.

By contrast, after a recent Amtrak train derailment in the US (Tuesday 12th May) on the line from Philadelphia to New York in which eight people were killed, the president and CEO, Joe Boardman, sent all passengers (including me) an email on Saturday 16th May which included this:

“With truly heavy hearts, we mourn those who died. Their loss leaves holes in the lives of their families and communities. On behalf of the entire Amtrak family, I offer our sincere sympathies and prayers for them and their loved ones. Amtrak takes full responsibility and deeply apologizes for our role in this tragic event”.

If my confidence in Amtrak had ever wavered, this response restored it immediately. This is a company which recognizes failure, apologizes and intends to address the problems. Everything is acknowledged publicly. Bereaved families were not having their grief aggravated by being treated to secrecy and silence.

These are both great tragedies for all affected by them, and I do not wish to diminish them by making inappropriate comparisons.  However, universities could learn something from this. In the UK and the US, university managers increasingly view their institutions in terms of business and markets. No slur is allowed to attach itself to the university’s name. Social media policies, and behaviour codes on ‘civility’ have emerged, which in some cases threaten academic freedom and academic careers.

Civility initiatives have been defended by Nigel Thrift, vice-chancellor of Warwick University, in 2012 in the US Chronicle of Higher Education, so his reaction to apparent insubordination from Professor Thomas Docherty in 2014 was not surprising. Docherty’s offense was reported to be nothing more than sighing and making ironic comments in meetings. What the academic community suspected, was that his views on managerialism and audit culture in universities had attracted the rage of the university’s senior management. Professor Docherty spent an uncomfortable nine months suspended from his position, unable to use campus facilities or correspond with students and colleagues. All charges were later dismissed, and Professor Docherty returned to work. Like footballers mobbing a referee after a controversial decision, Warwick management’s strategy was not intended to change a result, but to exact more cautious behaviour from him and others in future. Beware the next time you appear to be critical of your university, or even the state of higher education, especially if you cannot draw on prominent academic standing, and support from the Times Higher, among many others. But reputational damage has been done to the Warwick ‘brand’; many in the academic community now question their commitment to academic freedom, and freedom of speech.

This mistrust was heightened when, in July 2014,  a lawyer in the firm acting for the university, SGH Martineau, posted a blog piece which seemed to validate macho management techniques ‘pour encourager les autres’.  Titled ‘getting your teeth into high performer misconduct’, it was offensive for many reasons, not least because it made an explicit analogy between the conduct of star footballer Luis Suarez, who had recently bitten an opponent during a game, and academics who ‘damage their employer’s brand’ by their outspoken opinions. It is worth quoting from the piece in full:

“Universities and colleges may, equally, encounter high performing employees who, although academically brilliant, have the potential to damage their employer’s brand. This could be through outspoken opinions (where these fall outside the lawful exercise of academic freedom or freedom of speech more widely) or general insubordination, e.g. a failure to comply with the reasonable requests of an employer, or other behaviour such as bullying or harassment of colleagues. Irrespective of how potentially valuable these employees may be to their institutions, the reality is that, in consistently accepting unacceptable behaviour, institutions may be setting dangerous precedents to other employees that such conduct will be accommodated. From a risk perspective, it is also much harder to justify a dismissal, or other sanction, if similar conduct has gone unpunished before”. []

I may be mistaken, but I recall that the piece was edited, and the clause in brackets added, after a Twitter flame fest broke out between an outraged academic community, and SGH Martineau. The amendments just cemented the impression that academic freedom was merely an inconvenient afterthought.

Another current controversy drawing press attention is the climate of sexual harassment on many UK and US university campuses. In the US, President Obama and Vice President Biden have made personal commitments to ending the rape culture on campus. Even in the face of presidential support, it is still not uncommon for universities to try and block attempts to research the issue, as recently happened at the University of Oregon.

Here in Britain, ‘laddism’ is seen as part of a campus culture which emerges in a climate of hyper-masculinity, sports teams and alcohol use (Phipps and Young 2015). It is almost always pack behaviour and characterised by sexism, misogyny and homophobia. There is no doubt that it forms a hostile environment for women students. A 2010 report funded by the National Union of Students, Hidden Marks, found that one in seven women had experienced sexual assault, and 68% had been verbally harassed. The National Union of Students conducted another survey in 2015 which found one in three students had been sexually assaulted or abused while at university

This issue has made the news consistently in the intervening five years, however, a report which featured on the front page of The Guardian on 25th May 2015 revealed that fewer than half of the Russell Group universities systematically record rapes, sexual assaults or sexual harassment.

Despite this accumulation of evidence, Dr Wendy Piatt, the director general of the Russell Group, is quoted in the article as saying: “Russell Group universities take the issue of any kind of harassment, abuse or violence against women extremely seriously indeed. Our institutions have robust policies and procedures in place to deal with these matters, because ensuring student safety and wellbeing is extremely important to us.” []

It might be more accurate to say that universities have robust procedures to make sure that instances of sexual assault and harassment are kept well buried. Even bystander intervention campaigns can make senior managers nervous in case their university is seen as a ‘rape campus.’ It is ironic, given the emphasis we are all supposed to place on the much-vaunted ‘student experience,’ that this is subordinated to reputational concerns whenever there is a conflict.

Universities are on morally shaky ground in persisting with a stance of denial and negligence, but increasingly they are legally vulnerable as well. In the US, some women students are filing discrimination suits under Title IX legislation (the 1972 law that prohibits sex discrimination at colleges that receive US federal funds) claiming that universities do not provide them with a safe learning environment. Students at Xavier University of Ohio and  Harvard University Law School are the most recent to have successfully pursued claims to force universities to institute training and response to sexual assault.

In my own work as a lecturer in linguistics, I read projects and dissertations every year that draw on recordings of actual conversations among students. I am only too aware of male students’ discourses of rape and sexism, as well as racism. This year, I have learned a lot from being part of a ‘respect and consent’ group at my university as we move to create culture change and a better environment for all students, bolstered by an explicit code of behaviour and campaign of education. All UK universities would do well to take the issue seriously and follow the example of those universities who take women’s safety seriously.

As a general point, I hope university managers are learning something about brand management that they won’t find in business school leadership courses. Universities are charged with defending academic freedom, freedom to challenge authority, and the responsibility to provide a safe environment. The cases I have outlined have shown that an over-concern with reputation is, paradoxically, more than likely to lead to damaging your reputation. The lesson is – do an Amtrak, not a Thomas Cook.


McVeigh, Karen. 2015 Top universities fail to record sexual violence against women. The Guardian. Accessed May 24th 2015.

NUS report: That’s what she said: women students’ experience of ‘lad culture’ in higher education. 2015.

Phipps, Alison and Young Isabel. 2015. Lad culture in higher education: agency in the sexualisation debates. Sexualities. ISSN 1363-4607

Critical university studies, discourse and managerialism