Category Archives: HE policy

Resist, Insist, Persist

One of the things I miss about my life in a university is the recommended reflective feedback sessions with my managers after each staff development training. I’m sure they don’t feel the same. But to continue the habit of a lifetime – and with very sincere gratitude – I want to thank everyone who read this piece I wrote in the Times Higher (2nd March 2017)  Thanks especially to those who commented, retweeted, quoted, DM-ed, emailed or hollered. Reactions fell into largely three categories: congratulation, commiseration and corroboration.

First the congratulations, and there were some lovely QTs which featured adjectives like inspirational, insightful, courageous, powerful, excoriating, remarkable, brilliant. These all made me blush, but these linguistic judgments of esteem and veracity fuelled me with more determination to keep writing.

From the US, a Professor Emerita wrote: “It’s very interesting to read the comments of many here, who speculate on how to rebuild or create a better system, and whether or not that is possible. With such minds as I see among you and your friends, dear Liz, there seems to be a great deal of possibility–and it’s exciting to imagine what very different ways of learning you may bring into being in the coming years”.

From a fellow blogger with a keen critique of university policy and implementation:  “You’ve become a lodestone for us all as an example of the ethic of academia, and how difficult it’s becoming to behave ethical in the current structures. You’ve helped me personally and intellectually Liz – I hope there’s a lot more to come”.

Other colleagues offered commiseration and expressed their own sense of disenchantment.

This was from a union colleague: “It is getting increasingly difficult for me to experience the constant trampling of basic professional ethics. I was disgusted to hear what had happened to you, Liz, after sharing your views on performance management. Sadly so few academics today understand the value of academic freedom, in part because they are not doing work sufficiently controversial as to require its protection”.

It was consoling to hear from a few pro-vice chancellors. One wrote: ” Very sorry to hear about your recent experiences. You seem to have been treated very badly. Your Times Higher article is really effective in keeping these issues on the agenda”.

From another: “When I read your article in the THE this morning I was overwhelmed by a great sadness to think that you had left your university following an absurd ‘disciplinary’ process.  This evening I read it again and I am furious to imagine what nonsense you must have been through.  I am just sorry I did not know about it while the process was in train, to provide solidarity and counsel.  This injustice will remain a running sore as long as it is acceptable to think that the best way to do academic work is outside the academy.  This wrong must be righted.  It is just such a pity that academics and our union (don’t make me laugh) is so bad a mobilising around truly important issues”.

Two readers told me privately that they had broken down in tears after reading my piece. When a person suffers burnout and emotional distress, their own empathic reactions to another’s plight can be overwhelming. This I know from experience.

Others corroborated my analysis by sharing their own experiences of audit culture in universities. The panoptical nature of the surveillance, the punitive actions that accompany it, and the often unattainable targets demanded, all add up to stress, despondency and mental illness. One colleague pointed towards a future of algorithmic performance management of the sort identified at Amazon in the New York Times expose.

“Here at [Russell Group University] we have [XX company] coming in soon which allows real time micro level performance management via ‘dashboards’ recording all data on all staff for the duration of the ‘staff member’s life cycle’…. I guess that means electronic module evaluation feedback to save processing time, being added to H-index etc. data. Although this is denied by managers, part of the purpose with this is to strip out middle management and allow central / senior management to set targets for ‘teams’ and saying they can ‘liaise’ with the ‘core member’ if they need resources to achieve their targets, which really means, ‘we have direct control, total ‘transparency’ and can get rid of teams that ask for too much”.

A colleague in Australia DM-ed: “I’ve just come out of a traumatic couple of weeks in which I was asked to write a self-evaluation report identifying and quantifying my value to the university. I’ve been told that unless I can come up with some ‘low-hanging fruit’ in the short term my days are numbered etc. And I am – by any measure – a highly productive academic with millions of dollars in grant money, a plethora of publications etc”.

A colleague at another Russell Group University wrote: “The problem seems to me to be that the institution’s demands for compliance wreck our intellects (and our resolve and resilience), while stamping on us with disciplinary power whenever we point this out”.

This theme was best summarized in a tweet by one respected commentator (well, I respect him, and if he’s reading, I’d be really chuffed if he’d follow me on Twitter. Like what else do I need to do??):  “Powerful piece by @lizmorrish in THE today; something is going horribly wrong with way academic staff are managed”.

A colleague at yet another Russell Group University which has had its own issues with metrics reflected on the influence of a talk I gave in November 2015: “Thanks too for the mention of our success in [Russell Group University] in resisting a ghastly outcomes-based performance-management system last year. You came and gave an insightful and inspiring critique of it to a UCU branch meeting, which provided us with courage and the intellectual tools to tackle it. I am sure you will continue to play such a role. How can those of us still in universities support you at this time and going forwards?”

After such a show of appreciation, I can honestly say that I do feel supported – anchored, actually – in a community of scholars from which my former employer thought I had been ejected. I am fortunate to be able to continue writing, blogging and reaching out on Twitter and hopefully connecting and influencing that way. So in that sense I am ok. It’s the rest of you left behind that I worry about, so let me make some suggestions.

The people who need our concerted support are those whose academic freedom is compromised because their contracts are temporary or zero hours forcing them into the hire and fire economy of contingent labour. They dread questioning authority and have no real autonomy either in the classroom or outside of it. Thankfully, our union is campaigning on this issue. We must put pressure on universities to take measures for sustainable careers post-PhD.  The University of Birmingham is making a start with its research fellowship scheme – 5 years of research followed by a lectureship. This is a positive development.

We must talk to colleagues and students about the effects that work-related stress is having. When I was still working at a university, I was often sought out by colleagues for these conversations because my research and stance offered reassurance that it was the system and structures which were the problem, not the individual. This remains the case, despite all the wellbeing workshops and employee assistance programs being implemented across the sector. Unless we challenge management-by-metrics, academics will continue to get ill. This recent article in the Guardian Academics Anonymous addresses the embargo on talking about stress and mental health in universities. As my Times Higher piece reveals – there can be penalties for breaking the code of omerta, but we must.

We must resist collapsing our academic identities into a set of data points and spurious proxy metrics for ‘performance’. Let’s not talk about being REFable, or incorporate our h-index into our email signature. Instead, resolve to have conversations about interesting research, and how we add to it or want to integrate it into our teaching.

We can put pressure on our institutional managers to sign up to DORA: the San Fransisco Declaration on Research Assessment. Imperial College has just become a signatory, and several UK research universities have now committed to this pledge not to use journal-based metrics in hiring and promotion decisions.

Universities are held in thrall by the REF, and will presumably be so again with the TEF as long as government uses these mechanisms to control funding. It is sometimes hard to differentiate unavoidable external constraints from gratuitous control of academics’ behaviour. But if we empty academic careers of autonomy, then we risk being left with universities full of dressage ponies.  Let’s resolve to use our own judgement in our ‘self-directed research and scholarship’. We owe that to our students and our disciplines.

We should reject the damaging discourse of ‘excellence’ that has invaded every corner of universities. This is critiqued in an excellent paper by Moore et al. (2017). As the authors point out, excellence is not a discoverable quality. It is, of course, a fiction. At best it is a discursive strategy to normalise the achievements of the most talented and ambitious academics and make everyone else seem deficient by comparison. At worst it is a smokescreen for what Joyce Canaan calls ‘a culture of crappiness’. Moore et al. recommend that we retrain ourselves to evaluate our academic endeavours in terms of soundness and capacity.

And lastly we must ask our union branches to monitor any rise in disciplinary actions against colleagues, and scrutinize the effects on academic freedom, or rise in fear of inappropriate reprisals. There is a perception that there has been more frequent recourse to these procedures, but we need evidence and consistent monitoring.

I am grateful to Agnes Bosanquet who blogs at The Slow Academic. She writes about small targeted acts of resistance (STARS). In a citation she gives these examples: “Individuals were deliberately maintaining their research interests in defiance of perceived [audit]-rewarded tends; departments were actively pursuing collegial rather than competitive practices.”

These are all things we can do individually and collectively to resist the erasure of our academic autonomy by audit and the limits that discourse sets for our sense of achievement.

Some of my colleagues asked me why, unlike Marina Warner in this hard-hitting piece and another,  I refrained from naming the institution I left. The first reason is because my critique has never been intended to single out one institution – the problems are quite manifestly sector-wide. The second reason is more complex. The managers who chose to pursue me with disciplinary action will recognise themselves in the piece. The postmodernists among us would call this interpellation, and in queer theory, individuals are interpellated by shame if they respond to a hailing. They are hardly likely to step forward and claim their ignominy by objecting, in the same way they shut me down the last time. On that occasion they isolated me with a bond of silence. Now I have turned the tables and gagged them. One small targeted act of resistance.

Ten Myths and a Truth from the TEF: Reading the White Paper

Although the Higher Education and Research Bill is still going through parliamentary scrutiny, the Teaching Excellence Framework is about to be implemented and yet we do not know for certain what its effects will be, or even which institutions will enter into it. On the 2nd of December 2016, the same day as students at Warwick University went into occupation against the TEF , the chair of the TEF, Professor Chris Husbands,  published a blog piece entitled Busting five common myths about the TEF. A welcome addition to the critique, I thought, but I felt as though we were reading different documents.  I have been working on Chapter 2 of the White Paper (TEF) and so I checked some of Jo Johnson’s claims against evidence from some of the other publications I have been reading recently. Concealed within the pages of Jo Johnson’s White Paper, Success as a Knowledge Economy, May 2016,  are quite a few contested propositions and ten more myths which Chris Husbands has overlooked.

We hear much of how political discourse operates in a post-truth culture, but one of the key strategies of persuasion is via presupposition – an statement whose truth is assumed without substantiation. Another trick is to make syntactic linkages between concepts which then acquire the appearance of logical relationship. We find both of these demonstrated in the White Paper.

Below I outline myths (quotations and presuppositions from the White Paper) and responses based on evidence and reason.

Myth 1: There is a problem with ‘lamentable’ teaching quality in universities.

Response: There is no evidence presented to sustain the claim. Use of an inflammatory adjective installs the presupposition.

Myth 2: Students cannot make informed choices….These decisions are significant factors in determining a student’s future life and career success, so it is crucial that they represent sound investments. We need to make sure that students have access to the best possible information to make choices about what they study, and the benefits that they can expect to gain from those choices.

Response: Students have a lot of choice of courses, and they make up their own minds by consulting websites, alternative prospectuses, going to open days. There is even metricised data from Unistats  (comparison site which evaluates NSS scores, employment data and graduate salaries – exactly the innovation Jo Johnson thinks the TEF will deliver) and from league tables.

Nouns like ‘investment’ can also operate as presuppositions as the concept is assumed to be inevitable and universal.  ‘Investment’ is presented in crudely financialised terms as ‘return on investment’ or ROI, which presupposes that students are primarily concerned about future earnings. No evidence is presented to substantiate this, even in the face of students continuing to apply for courses where relatively low salaries are likely upon graduation e.g. nursing, creative arts, education, agriculture. We note that ‘investment’ is a polysemic (multi-meaning) term used to reference the expending of economic capital, and emotional/ intellectual capital by the individual.

Myth 3: Robust, comparable information about the quality of teaching – and the components that contribute to it – is not currently available… That is why this Government will introduce the TEF and for the first time bring sector-wide rigour to the assessment of teaching excellence.

Response: A repetition of the presupposition that students do not already have access to this information. As stated above, it clearly is available. If it is not, why have we been pouring money into QAA, institutional reviews,  Hefce, etc. for all these years, if it has not had the effect of ensuring the quality and reputation of the sector? This architecture of quality assurance, though imperfect, has ensured that the UK is one of the most highly regulated and inspected sectors in the world.

Myth 4: The consumer organisation Which? has found that three in ten students think that the academic experience of higher education is poor value, and the issues raised by students in that research included the amount, and quality, of teaching they received, and the extent to which they are academically challenged.

Response: It is good to see a rare appeal to evidence, but perhaps the wrong conclusions are being drawn by the Which? study. This study by Steven Jones, Steven Courtney and Ruth McGinity proposes another interpretation: “Large fee increases mean that university is bound to be seen as exploitatively expensive by students. This does not mean they are dissatisfied with their courses or teaching quality”. In fact, the NSS scores nationally indicate that students are satisfied with their university experience. Can Jo Johnson make NSS a key metric, and then discount it, all in the same policy document?

Myth 5: Clear priorities of students while at university included: “having more hours of teaching”, “reducing the size of teaching groups” and “better training for lecturers”, but there is little information for prospective students on this in advance.

Response: As this study finds, effective student learning does not always emerge from ‘more contact hours’; in fact independent study is more valuable.   Learning may be the first casualty of a popularity-led evaluation like the NSS/ TEF.

Myth 6: Employers report a growing mismatch between the skills they need and the skills that graduates offer.

Response: A study reported in the Times Higher in 2015 shows that universities are doing a good job in developing the kind of skills which employers find useful and “UK employers are still among the most satisfied with their nation’s higher education system (giving it 7.3 out of 10, compared with a global average of 6.8).”

Myth 7: We need to ensure that our higher education system continues to provide the best possible outcomes. These come from informed choice and competition.

Response: This is a logical non-sequitur, but allows a lazy conflation of several unrelated concepts and assumes causality between them. The White Paper assumes that outcomes = return on investment = graduate salaries, and that these will be consequent upon informed choice and competition. Quality of courses, and choice for students, is more likely to emerge from imaginative cooperation between institutions. This would be an innovation worth pursuing.

This study by David Morris of Wonkhe analyses the government’s Longitudinal Earnings Outcome (LEO) data. There are a number of departures from the outcomes-require-competition myth. Prior attainment, i.e. A Level performance, makes a huge difference to graduate earnings, regardless of subject studied.  This raises a question about ‘learning gain’ – also a concern of the White Paper. I’m sure this will present itself as another cudgel to beat less-favoured universities with. However, Morris’ study also identifies a gender gap and a race gap for earnings, which is far less consonant with a learning gain/ value-added analysis.

Myth 8: By removing student number controls and making it easier for new providers to enter, we will create the conditions that will allow choice and competition to flourish. But what is also needed is the information to allow students to determine where the best teaching can be found.

Response: The answer to quality enhancement, we are expected to believe, is the entry of new providers in order to create ‘competition’. Except the new providers will not be expected to fulfil all the expectations that publically-funded universities are expected to address. As this article makes clear, as new private providers have emerged in strength in South America, especially Argentina and Chile, they have not been engaged in research. This, argues the author – Carolina Guzmán-Valenzuela, restricts the number of qualified PhDs who are able to take the higher education system forward.

Myth 9: The Government believes that excellent teaching can occur in many different forms, in a wide variety of institutions, and it is not the intention of the TEF to constrain or prescribe the form that excellence must take. What we expect though, is that excellent teaching, whatever its form, delivers excellent outcomes.

Response: The TEF will have criteria, and metrics, so how can the White Paper say that the form of excellence will not be prescribed or constrained. In fact, that is exactly what will happen as institutions align their priorities precisely to those criteria – which as the statement makes clear, are in any case based on the proxy ‘outcomes’ of NSS scores, retention and most importantly graduate salaries which are high enough to pay back all the money the government has lost in its ill-advised restructuring of HE finance towards what are, in effect, individual student vouchers.

Myth 10: Perhaps the biggest myth of all – as Jones, Courtney and McGinity point out, is Johnson’s claim that the TEF will strengthen the position of students.  It will not – and indeed, the NUS has voted to disengage from TEF. Evidence shows that co-opting students as consumers is damaging to educational experience.

A truth – a veritable truth: There is of course more to university than financial gain, but the idea that excellent teaching occurs in a vacuum, independent of its impact on students’ future life chances, is not one we can or should accept.

Response:  There is a nice hat tip to other justifications of HE, but immediately we see the counter-narrative remains in place with co-reference of outcomes with financial gain, disguised as ‘life chances’. The presupposition is that the most significant outcome of higher education is employment, but as this study shows, economists have often found that education has benefits for society beyond those of the individual – for example in terms of volunteering, social trust, better citizenship (lower crime).

 

Whatever does ail the higher education sector in the UK, the TEF spreadsheet will not fix it. Much more likely is that the government will recruit ‘consumer choice’ as a disciplinary tool, overlooking the needs of scholarship, local economies or student interests, and possibly serving as licence for university closure. By allowing this false reasoning to go unopposed, we risk losing quality, opportunity and reputation within the sector. Here is a link to the Convention for Higher Education website which has some key resources for opposing the TEF and the Higher Education and Research Bill. Organise, and support students in their refusal to co-operate with the TEF and NSS as long as it threatens to raise their fees, waste millions of pounds of their ‘investment’, threaten the reputation of their courses and distort the priorities of universities away from good teaching and research.

 

A Stern Talking To ?

The Stern Review has appeared and HE wonks are rushing back to the laptops they had just powered down for the briefest of summer holidays. So here are my thoughts, based on a skim reading of the report, and also the helpful comments on Twitter from the following: Emilie Whitaker @Dr_EmWhitaker , Steven Jones @StevenJones_MCR David Wright @WrightDW  Daniel Grey @djrgrey  Mike Ratcliffe @mike_rat and James Wilsdon @jameswilsdon.

It is clear that Stern endorses the continuation of the REF. This is no mutinous denunciation of research assessment. But then that was never in the terms of reference with its forecast of ‘future iterations of the REF’. He has, though, proposed fixes to escalating costs, institutional gaming and some of the impediments to providing a true picture of UKHE research, such as requiring a demanding output-impact case study ratio and ‘selectivity’ of individuals submitted for assessment. There is also a welcome focus on support for interdisciplinarity which many have felt was overlooked in REF2014. Derek Sayer’s important criticism that the REF is a parochial and navel-gazing exercise is addressed; at last, international researchers are to be invited onto panels (Para 109).

One of the stated imperatives of Stern is to reduce the workload of the exercise. He recommends a strong reduction in the average number of ‘outputs’ submitted per faculty member (Para 69) which at this point is looking like a baseline of 2, or even fewer. There will still be sampling of outputs by unit of assessment (Para 71). One more priority is to reduce gaming of the system, and to this end Stern has recommendations about who owns ‘outputs’, and an end to ‘selectivity’.

The most radical of the recommendations in terms of its implications for current practices is that there should be no portability of research ‘outputs’. Stern feels that they should remain returnable by the HEI which ‘helped to produce’ them. This raises the issue of who owns research ‘outputs’ and is guaranteed to cause indignation among the more mobile academics, and also Early Career Researchers who may have had a series of short-term contracts at different universities. It would, however, reduce the likelihood that avaricious institutions would poach individual researchers if they were entitled only to ‘outputs by the individual that have been accepted for publication after joining the institution’ (Para 74). There is no doubt that this will hit the perceived ‘marketability’ of individual academics, but perhaps Stern has rather harshly assumed that the only reason academics leave universities is because of their own ‘rent-seeking behaviour’.

Recommendation 1 is that all research-active staff should be returned in the REF, noting that ‘exclusion and the associated stigma are being driven by factors that are not wholly related to the quality of an individual’s research contributions and potential’ (Para 64). While this is commendably inclusive, and recognises that excellence in research may be found anywhere, it might also act as an incentive for universities to reassign staff to career-limiting teaching-only contracts. This fear is mitigated by the recognition that the proposed TEF looks favourably on research-led teaching (Para 112). Like other high-level communications and policy documents, the Stern Review also presupposes the implementation of the TEF, even though (in July 2016) this proposal is still being debated in parliament.

There seems to be some sanity at last on the notion of research impact. This should be based more generally on research activity, and not dependent on particular ‘outputs’. Also, assessment of impact need not be confined to socio-economic impact as it was for REF2014. Stern recommends that it ‘should also include impact on government policy, on public engagement and understanding, on cultural life, on academic impacts outside the field, and impacts on teaching’ (Recommendation 7).

From my reading, I felt my antenna twitch at a couple of potential Trojan Horses. Stern allows scope for the use of metrics, while cautiously invoking James Wilsdon’s ‘Metric Tide’ report of a year ago. Once metrics are allowed to stand as proxies for the health of a unit, then inevitably this lends permission for greater infiltration.

Indeed, the primary way the research environment will be assessed will be via metrics (Para 48). It seems perverse not to allow for a qualitative statement in this particular area. One wonders how an environmental template based on metrics will facilitate the ambition to capture ‘the contribution that its academics make to the wider academy (‘academic citizenship’)’ (Para 88). Recommendation 4 states that panels should continue to assess on the basis of peer review but may supplement this with metrics, as long as they are transparent about their use (Para 105).

Sadly, there is no recommendation to commit institutions to become signatories of the San Fransisco Declaration on Research Assessment. HEFCE is currently a signatory, although in a rather insouciant way. Instead, Stern invokes an aspiration to standardise the metrics supplied to various bodies whose incompatibility currently stands in the way of more widespread use. There seems to be an invitation to UKRI to become the overlord in this integration of research metrics (Para 107). Will they, too, sign DORA?

The REF needed a stern review, and it has got it. It offers enough clarity to subdue even the most feverish of vice-chancellors for a weekend. It seems to be informed by principles of parsimony and lean management in contrast to the speculative profligacy of the White Paper’s TEF. If we must live with the REF, an ‘iteration’ whose more rapacious and distorting inclinations have been curtailed is preferable.

TEF Times: 2nd Reading of the HE Bill

In July 2016 we are contemplating a new period of instability for universities in the UK, and with the passing of the 2nd reading of HE Bill, things could quickly get a lot worse. The EU Referendum result has already created uncertainty regarding the future of much of our research funding. It seems there is much uncertainty at the top of UKHE: Universities UK (‘the definitive voice of UK universities’) has asked for the government to press the pause button on HE reform , Meanwhile, the vice-chancellors of Nottingham Trent and Exeter Universities argue for forging ahead with reform and the implementation of the Teaching Excellence Framework (TEF).

Maddalaine Ansell, CEO of the University Alliance, appears to agree with the latter in her prediction that the HE Bill will take the sector to calmer waters.  Ansell’s premise is that there will be a benefit from having all legislation relating to HE encompassed in one piece of legislation: The Higher Education and Research Act 2017. Except it won’t, of course. A moment’s reflection allows us to list student loans, students with disabilities, and academic freedom – all of which have separate legislation. Additionally, Ansell appears to overlook the added complications of teaching and research which will now be overseen by different government departments since Theresa May’s July ministerial reshuffle. It is a complicated picture, and I cannot see any advantage to deepening it.

Let us remind ourselves just how disruptive these changes proposed by the White Paper entitled Success as a Knowledge Economy (SKE) will be. They include an invitation to new private ‘challenger institutions’ who may be granted degree-awarding powers more quickly than previous regulation allowed. There are changes proposed to governance, academic freedom and protections against arbitrary dismissal which appear to infringe the historic autonomy that universities enjoyed from government. A critique of the proposed changes can be found in an Alternative White Paper (AWP), authored by a group of concerned academics can be found here.

The most unnecessary and wasteful plan in the White Paper is for a Teaching Excellence Framework. This has been proposed to correct supposedly ‘lamentable’ teaching (AWP p28). The paragraphs which outline how this will work display some baffling logical linkages. Here are some of the assertions made in the paragraphs which outline the justification for the TEF:

  • higher education leads to better employment outcomes, but these outcomes are not consistent;
  • there is considerable variation in employment outcomes and employability amongst subjects and across institutions;
  • students often enter HE with little information to guide their choices;
  • students often say they would have chosen a different course;
  • the importance of students having access to a wide array of work experience opportunities;
  • a recent IFS study also found huge variance in graduate earnings depending on choice of subject and institution, as well as background;
  • higher average earnings mean that graduates make an important contribution to society through their tax revenues;
  • employers and HE providers working together on curriculum design, and graduates having the ‘soft skills’ they need to thrive in the work environment. (SKE p42).

Apparently, the answer to all of these is the TEF which they claim will raise teaching standards. From 2018/19, an award of excellent or outstanding will permit an HEI to increase its fees in line with inflation. Others, even those meeting expectations, will suffer various degrees of attrition and their students condemned to a ‘choice’ of an educational resource eroded by inflation.

The government remains confident that good teaching can be measured on an institutional basis, but the first point to emphasize is that these measures are, as the White Paper admits, proxies, not measures of good teaching which transpires in classrooms and other learning contexts.

“Such things can be measured: students assess their satisfaction with their courses, retention rates are a good proxy for student engagement, contact hours can be measured, employers choose to sponsor some courses, or work with some institutions, because of the industry-relevance of their offerings, and employment rates can be measured. Some of these metrics are of course proxies – but they directly measure some of the most important outcomes that students and taxpayers expect excellent teaching to deliver. And we recognise that metrics alone cannot tell the whole story; they must be benchmarked and contextualised, and considered alongside the additional narrative that can establish a provider’s case for excellence”. (SKE p46)

Secondly, nowhere in the White Paper is there any evidence of so-called lamentable teaching. In fact the published NSS figures show the opposite. Taken nationally, the average figure is extremely high at 86% (England 2015 NSS results) with a rather small range of scores. So why, asks Dorothy Bishop, is there any need for a TEF?

It is hard to avoid the implication that there is likely to be a shift in the direction of prioritising graduate earnings, and indeed, it is one of the proposed measures as the TEF moves towards “a more granular and informative assessment of graduate outcomes” (SKE p48). Possibly the best expose of this misguided proxy measure was the study published in April by the IFS. It demolishes graduate salary as a metric, with its finding that “Graduates from richer family backgrounds earn significantly more after graduation than their poorer counterparts, even after completing the same degrees from the same universities.”  Although this study is acknowledged in SKE, the logic is not absorbed. If we can assume that excellent teaching will not be restricted to more socially advantaged students, what relevance is there to measuring graduate earnings? We can detect an implicit threat in the White Paper that the government may seek to pressure universities to close courses which do not deliver the right ‘outcomes’, i.e. graduates who are able to earn enough to pay back the cost of their student loans. That, then, is the real purpose of this metric. Purely ideological – your graduates don’t pay back – your course is closed.

“In creating the OfS, the regulation of higher education will be restructured, shifting from an outdated, top-down model of a funding agency to a market regulator clearly focused on the student interest. We will give the OfS an explicit duty to promote choice and competition, which will increase quality and efficiency in the sector, and will expect the OfS to work closely with the Student Loans Company and Government to ensure the decisions it takes have regard to affordability and deliver value for money for the taxpayer”. (SKE p63)

In 2017/18 the TEF will be run on a voluntary basis. A ‘provider’ can opt in, presumably if it wishes to establish a good reputation for teaching. A mock league table of benchmark-adjusted metrics published by the Times Higher showed that the Russell Group universities were eclipsed by a Midlands triangle of Loughborough, Aston and De Montfort universities. But this could also be part of the script. The government is creating the conditions whereby the Russell Group flounce out of the TEF and follow the incentives towards privatisation. It is only a matter of time before the elite universities follow their counterparts in Australia and start charging variable fees which will have nothing to do with teaching quality and everything to do with accrued reputation – something which the White Paper claims it wishes to dismantle. Rather than providing concrete information on which students can base their choices, this uninformative snapshot will leave students confused between choosing between the dodgy dossiers of established reputation and the imposter proxies of the TEF.

The TEF will do nothing to increase good teaching, curtail bad teaching or provide students with any more guidance than they already have. And if the REF is anything to go by, it will involve escalating costs and a scale of wastage which makes older, experienced academics weep with regret at what could be achieved if only the money were spent wisely. The cost-benefit analysis is provided by Dorothy Bishop here.

Universities have gone along with the REF because (up to now at least) there were reputational, even if few financial, gains to be won. The TEF allows for little financial gain, and also looks to be repeating some of the reputational mistakes of the early QAA subject reviews which denounced some subjects as failing. The TEF, even when it launches its disciplinary-level ‘granularity’ will not be a ‘game changer’.

Even though universities now have the tools to immediately individualise TEF scores of student satisfaction, nobody is going to be poached by an HEI for their superior teaching scores. Similarly, I would imagine that few academics will be to be tempted to move to a stronger teaching department. And bear in mind, academics have limited agency to affect outcomes such as retention, student satisfaction and employment. Students may be very satisfied with individual teachers, while perceiving elements of the course to be disappointing, funding to be inadequate, accommodation too expensive or the claims of family or paid employment to be stronger.

For universities it is another hurdle to be surmounted. A promised tuition rise in line with inflation will be quickly consumed in the arms race to enhance the institutional image. But the government’s nudge unit will clock up another win as soon as it achieves the desired outcomes; privatisation of an elite tier of universities free to charge whatever they wish, and perhaps, the closure of a few universities which have widened participation, but failed to compensate for the calculated upward distribution of wealth which has been part of the neoliberal project. Whether the HE Bill is creative disruption or reckless joyriding remains to be seen.

Normalising immorality

My last blog post was about the perverse incentives currently circulating in universities which lead good academics to do bad things. I cited studies which indicate academics may lie to get research grants, selectively present data confirming a hypothesis or exaggerate their findings to get published. This was read by one commentator as blaming academics for merely responding to the conditions which are necessary to keep their jobs. And she didn’t care for the appropriation of the Trump analogy either.

OK, time to get right back into the water. My point was that, primarily, it is the structures within which academics work that are to blame. Governments send down their edicts, and universities seek to maximize their opportunities within them. But there must also be some degree of agency which we can all exert in defiance of corrupting structures. I want to state why it is unacceptable for any of us to overlook dishonesty and the undermining of legitimate process, and why we need to act collectively to stop it.

I’m becoming quite a fan of Rowan Williams. For one thing he examines the dangers of tolerating hypocrisy and unethical behaviour. For another, he does not speak well of Donald Trump, so he’s my man. In March 2016 he spoke locally about ethics, morality and empathy. His argument was that when people behave unethically, it does not mean they are devoid of empathy; in fact, the reverse. Those who perpetrate causal cruelty achieve their result precisely because they recognise what they are doing, and understand the extent of their victims’ suffering. These are unusual people, but how do they manage to get away with this kind of evil? How do bad things happen in what seems like a good institution?

Edmund Burke wrote “all that is necessary for the triumph of evil is that good men do nothing”. This, says Williams, in a recent article in New Statesman, gives evil far too much credit and agency. He believes that evil creeps upon us in rather a different way, more like a perfect storm that a strategic plan. He outlines a slippery slope argument in which “we are at least half-consciously complicit”.

I am aware that some people find it offensive when others draw analogies between the Holocaust and more contemporary concerns in society. My own view is that it is irresponsible not to learn these lessons, and I imagine Rowan Williams would agree. Williams invites us to contemplate how complicity is constructed in a society, and he draws on the example of the Third Reich (which he calls “a masterclass in executive tyranny”) to illustrate his argument. In order to persuade a populace to collude in genocide, Hitler took advantage of some routine anti-semitism which had been normalised by the repetition of certain tropes and myths about Jews. In a re-reading of Hannah Arendt’s thesis of the “banality of evil”, Williams blames an “erosion of a sense of the ridiculous”. We may recognise ourselves in this characterisation; we find the ridiculous in the times when we have complied with an inducement to game a system, inflate a finding, or we have watched silently as others struggle with disproportionate demands. We remember those occasions when we have failed to confront the exercise of excessive power, and told each other, ‘this is over the top’. That is the ridiculous, and that’s when we need to act, because immorality starts with small concessions and by dint of permissiveness, end up overwhelming us. And that leads us to Donald Trump and his evocations of external and internal threats, barriers necessitated, and birth rights revoked. Williams sees him as an exemplar of someone “divorced from realism, patience and human solidarity”. Ridiculous, in other words, and our antennae should be twitching.

Williams ends: “For evil to triumph, what is necessary is for societies to stop thinking, to stop developing an eye for the absurd as well as the corrupt in language and action, public or private”. Let me be clear. I’m not setting myself up as some moral arbiter. I’m as flawed as the next person. But I do agree with Rowan Williams that it is imperative to watch out for danger signs in our own environment, and to act according to our consciences. To recognise when governments, corporations, behaviours have become excessive and harmful. It is about trusting our instincts over the hypocrisies we are asked to absorb. It is about having a clear sense of purpose and legitimacy. It is recognizing when the demands of the imaginary and the dishonest displace the integrity of doing your job. And it is about refusing to be silent when ‘theatres of cruelty’ (Couldry 2008) invade your very humanity. Rowan Williams has certainly not restrained himself from denouncing a “new barbarity” in the de-humanising language and expectations circulating in UK universities.

Since reading Christabel Bielenberg’s powerful account of her family’s anti-Nazi resistance during the 2nd World War, I have been preoccupied with what Williams calls “moral luck” – “the fact that some people with immense potential for evil don’t actualise it, because the circumstances don’t present them with the chance, and that some others who might have spent their lives in blameless normality end up supervising transports to Auschwitz”. Perhaps also the converse must be true – that people with the capacity to resist immorality and corruption are not called upon to do so. But that seems unlikely to me, given the moral forcing ground that surrounds us in contemporary academia. Most of us know when things are not right and we are being manipulated into unethical behaviours. But it is easy to lose our perspective when coerced by threats of losing our jobs or punitive consequences for not meeting ‘targets’. As Williams writes, all it takes is “the steady and consistent normalising of illegitimate or partisan force, undermining any concept of an independent guarantee of lawfulness in society”. There are no accidents of immorality – there are choices. The choices may be unwilling, but please let’s start standing up to misuse of power, authority and expertise before we start accepting it as the new normal, and it empowers the next step towards dishonesty and corruption. Because if we let go of academic values of honesty, integrity and fearlessness, then along with them go academic freedom and a little bit more of our humanity.

References

Bielenberg, Christabel. 1968. The Past is Myself. London: Corgi Books. https://www.amazon.co.uk/Past-Myself-Christabel-Bielenberg/dp/0552990655

Couldry, Nick (2008) Reality TV, or the secret theater of neoliberalism. Review of education, pedagogy, and cultural studies, 30 (3), pp. 3-13. http://eprints.lse.ac.uk/52405/1/Couldry_Reality_TV_secret_theater_2008.pdf

Morgan, John. 2015. Rowan Williams on higher education’s ‘inhuman and divisive’ jargon. Times Higher. January 29th. https://www.timeshighereducation.com/news/rowan-williams-on-higher-educations-inhuman-and-divisive-jargon/2018188.article

Williams, Rowan. 2016. A nervous breakdown in the body politic. New Statesman. 1st May. http://www.newstatesman.com/politics/uk/2016/05/nervous-breakdown-body-politic

Are we seeing the rise of the Trump Academic?

A post by Mark Carrigan caught my imagination a few weeks ago. It was about ‘networking’ with its usual connotations of “insincerity, instrumentalism and general creepiness”. We can all recognise the ambitious researcher at the conference who is anxious to advertise their own work while affecting interest in the keynote speaker’s presentation.  It resonates with my current work on academic self-promotion via university profile pages. And I start to wonder, is a new academic habitus is beginning to emerge?

There are a lot of incentives for a young academic to develop chameleon-like characteristics. In an era when many are forced to spend years on short term, casualized contracts, your advancement increasingly depends on accumulating the right kind of academic accolades. I refer to this as the ‘Bisto kid’ approach, whereby you learn to sniff the air and follow the smell of money, because money now equates to academic regard. Pursuing grants will be your ticket to a permanent job, time for research and access to international conferences. Without it, you may be left alone with your passion and originality to generate an unfunded, and therefore marginalised, research program.

The consequences for academic freedom have been obvious for some time, but other problems are now starting to become apparent, particularly in the sciences. There have been an unsettling series of stories on the same theme in the Times Higher and in the blogosphere recently.

A study conducted in the UK and Australia indicates that academics regularly lie to inflate the predicted impact of their research (Matthews 2016 a). One professor in Australia is quoted as saying “It’s really virtually impossible to write an ARC (Australian Research Council) grant now without lying.” It is harsh to condemn these colleagues when you consider the ludicrous necessity of fabricating an impact statement. By definition, if the work has yet to be done, there can be no confident claims of either results or significance. The scientific community has collectively decided to acknowledge that if everybody knows it is absurd, then there is no moral hazard to trouble their consciences.

Two more significant issues have been discussed by Dorothy Bishop in her blog “There is a reproducibility crisis in psychology and we need to act on it”. If other scholars cannot reproduce the results of psychology experiments, this means the discipline has a problem which needs to be addressed, she argues. The other, related, problem is also methodological; the phenomenon of p-hacking – selecting experimental data after doing the statistics to ensure a p-value below the conventional cutoff of 0.05. (See also Nuzzo 2014 on p-values and reproducibility concerns). P-hacking is recognised as one reason for poor reproducibility of scientific findings, and the phenomenon is well explained for the non-scientist by David Colquhoun here.

Meanwhile, a number of academics are detecting an apparent bias towards positive results operated by journals in publication of scientific work. Simon Hatcher asks “Is something rotten in the state of academia?” and explains, “The need arises out of intense competition between Universities to compete for funding and for journals to publish the latest breakthroughs”, and he offers this insight into why this effect has arisen, “It increasingly seems that journals value themselves not by the quality of what they publish but by whether it is picked up on social or main stream media. As a result negative studies are much less likely to be published.”

Equally, in a world where academics are obliged to offer up each piece of work to be evaluated as internationally significant, world leading etc., they will seek to signal such a rating discursively. A study by Vinkers et al. 2015 in the British Medical Journal    uncovered a new tendency towards hyperbole in scientific reports. They found the absolute frequency of positive words increased from 2.0% (1974-80) to 17.5% (2014), which amounts to a relative increase of 880% over four decades. 25 individual positive words contributed to the increase, particularly the words “robust,” “novel,” “innovative,” and “unprecedented,” which increased in relative frequency up to 15 000%”). The authors comment upon an apparent evolution in scientific writing to ‘look on the bright side of life’.

It is rather more than that, though. This behaviour of extolling banal results is what is more commonly called ‘gaming the system’, and it is clearly being learned young. Tim Birkhead (2016) narrates an encounter with undergraduate students which opened his eyes to some disturbing attitudes:

Here’s a tiny but telling example of what we are now faced with. At the end of a recent seminar I presented on scientific misconduct, I asked the audience whether they knew anyone who had fabricated data for their final-year undergraduate project. Eighty per cent said yes. My jaw dropped and the expression on my face must have shocked the students since one of them came up to me and, touching my arm, said: “Tim, don’t be upset, we wouldn’t do this if it was real science.” This attempted reassurance only plunged me further into despair. I probed a little deeper and asked the students why they felt it was acceptable to fabricate data in a teaching context, but not in a “genuine” scientific investigation?

Xenia Schmaltz (2016) fast-forwards to the next stage of a young scientist’s career, when she questions “Good scientist or a successful academic? You can’t be both”. She encapsulates some of the contradictions in a simple table.

Good scientist

The behaviours on the left are those necessary for academic integrity, while those on the right may lead to academic preferment. Birkhead blames a culture that pressures academics into quantity and speed of publication, competition and league tables, a confirmation bias, a pursuit of safe science which might win scarce grant money, and the need to specify ‘impact’ before the work is even done. David Graeber (2012) writes, “If you want to minimize the possibility of unexpected breakthroughs, tell those same people they will receive no resources at all unless they spend the bulk of their time competing against each other to convince you they know in advance what they are going to discover”.

It is not just our research which we must ‘sell’. We sell new courses, conference, institutes, grant proposals, and, as I am finding, our academic selves on social media platforms such as LinkedIn, Academia.edu or ResearchGate.  The self-promoting academic has emerged hand-in-hand with the marketization of universities, so that now, the X Factor could be the next model for filling university jobs. Indeed, Jack Grove (2016) reports on a selection procedure for a postdoctoral position in a The University of Calgary’s medical school in which interviews were streamed to a live audience of 70 early career scientists who voted with 90% agreement for the same candidate.

The point of the Calgary exercise was to make the trainees more self-critical when showcasing their own skills, but it seems likely that such structures will produce a particularly narcissistic graduate, as this study (Matthews 2016 b) seems to predict.(For the original study click here). Examining traits of undergraduate business studies students, the authors found “more narcissistic students “thrived” with similarly self-obsessed instructors. These latter seemed to act as role models to the former, further legitimising their behaviour. By contrast, less narcissistic students did not thrive with the narcissistic faculty. Matthews’ article in the Times Higher asks: “As millennials enter the faculty ranks, will they be disproportionately narcissistic?” .

It is unfair to ascribe this behaviour to millennials.  Academia, often portrayed as the home of eccentric loners, has always had its share of self-important grandstanders. But current inducements are giving rise to a new phenomenon of the Trump Academic. Rather like The Donald, who is prepared to jettison previous party affiliations and beliefs in order to reinvent himself as the creation of Republican intolerance and bigotry, the Trump Academic will gladly set aside scientific passion or originality in favour of safe and fundable research. Their boastful certainties, unanchored in the truth, might allow the suppression of a negative result, or the inflation of some calculations to reach a threshold of significance. They may switch paradigms entirely in order to run with the popular crowd.

If you haven’t yet met the Trump Academic, you should probably get out more. Increasingly they will be hard to miss as the motivations coalesce around work which pleases governments, university managers and students. Now, even a permanent contract cannot guarantee the indulgence of ethical behaviour and academic freedom. Tressie McMillan Cottom writes of the rarity of great genius. When indisputable talent evaporates, all that is left is ‘the hustle’. The more that regimes of top-down interference chip away at academic authenticity and integrity, the further we are from that perennial pretension of ‘excellence’. We risk becoming a profession of self-regarding hustlers, changing our game plan at each new behest from above. Let’s count them: contract renewal, tenure, promotion, student evaluations, research selectivity, impact, employability. I was reminded of a recent article in The Guardian (Hattenstone and Allison, 2016), a profile of former Chief Inspector of Prisons, Nick Hardwick, who, on leaving the post said, “You shouldn’t do this job for too long because you get used to things you shouldn’t get used to”.

I hope those early-career researchers in Calgary aren’t saying that in ten years’ time.

  

References

Birkhead, Tim. 2016. Government policy is wrecking science. Times Higher. March 24th. https://www.timeshighereducation.com/comment/government-policy-is-wrecking-science

Bishop, Dorothy. 2016.  http://deevybee.blogspot.co.uk/2016/03/there-is-reproducibility-crisis-in.html.

Bishop, Dorothy. 2016.  http://deevybee.blogspot.co.uk/2016/01/the-amazing-significo-why-researchers.html

Mark Carrigan. 2016. http://markcarrigan.net/2016/04/04/how-to-network-without-chipping-away-at-your-soul/

Colquhoun, David. 2014. http://www.dcscience.net/2014/03/24/on-the-hazards-of-significance-testing-part-2-the-false-discovery-rate-or-how-not-to-make-a-fool-of-yourself-with-p-values/

Graeber, David. 2012. Of flying cars and the declining rate of profit.  The Baffler. 19. http://thebaffler.com/salvos/of-flying-cars-and-the-declining-rate-of-profit#left-menu

Grove, Jack. 2016 https://www.timeshighereducation.com/news/could-the-x-factor-be-a-model-for-filling-university-jobs Times Higher March 8th.

Hatcher, Simon. 2016. https://shatchersite.wordpress.com/2016/03/21/is-something-rotten-in-the-state-of-academia/  21st March 2016.

Hattenstone, S. and Allison, E. (2016). Interview with Prisons Inspector Nick Hardwick: ‘You shouldn’t do this job for long because you get used to things you shouldn’t’. The Guardian. 29th January. http://www.theguardian.com/society/2016/jan/29/prisons-inspector-nick-hardwick-interview  Accessed 1st April 2016.

McMillan Cottom, Tressie. 2016. There are thieves in the temple tonight.  https://tressiemc.com/2016/04/21/there-are-thieves-in-the-temple-tonight/ April 21, 2016

Matthews, David. 2016 a. Academics ‘regularly lie to get research grants’. Times Higher. March 9th. https://www.timeshighereducation.com/news/academics-regularly-lie-to-get-research-grants

Matthews, David. 2016 b. https://www.timeshighereducation.com/news/selfie-generation-grades-boosted-if-lecturers-are-narcissists  Times Higher. April 5th

Nuzzo, Regina. 2016. 2014. Scientific method: statistical errors. Nature. http://www.nature.com/news/scientific-method-statistical-errors-1.14700

Schmaltz, Xenia. 2016. Good scientist or a successful academic? You can’t be both. Times Higher. March 1st. You https://www.timeshighereducation.com/blog/good-scientist-or-successful-academic-you-cant-be-both

Fictions and Contradictions: Performance Management in Higher Education

Liz Morrish argues that the assumptions of performance management in higher education reside in the world of managerial fictions. It is a process riven with contradictions which require urgent rethinking.

Performance management has become a feature of the higher education landscape in the last decade. This definition is offered by Franco-Santos, Rivera and Bourne (2014), “At work, individuals are said to perform when they are able to achieve the objectives established by management. Organisations are thought to perform (or to be successful) when they satisfy the requirements of their stakeholders and are more effective and efficient than their competitors.” [my italics]

Recently, this document was circulated on Twitter. It is the assessment form required for performance review meetings at a non-aligned post 1992 university.

IMG_20150620_134505 (1)

It is important, as a first step to framing resistance, to recognise the presuppositions and ideologies enshrined in this document. One of the major presuppositions that underlies the use of performance/performing is that there are standard ‘key indicators’ of performance which are invariable. It is presumed that management will decide what these indicators are, and that academic staff performance can be objectively measured using them.

The most striking omission from the checklist above is any recognition of normative assumptions about academic work: that it involves teaching and research; that it is absorbing; that it involves insight, imagination, networking, diligence; that it is rather indefinable in scope, and quite possibly not a good candidate for this type of one-size-fits-all assessment.

The scope of the job, we assume, is contained within the unexpanded categories of Quality of Work; Quantity and Output. The contradiction lies in the under-informing of the designation ‘satisfactory’. In a working environment where assessments are multiple (NSS, REF, QAA, peer teaching reviews etc.), and ever more searching and fine-grained, what are we to take from ‘satisfactory’? It seems to leave open a return to a default ‘unsatisfactory’ if perceived under-performance in one of the measures should become unbundled from the totality. The semantic instability of the evaluative adjective ‘satisfactory’ means that staff will never be ‘performing’ perfectly. It will always be possible to claim that there are ‘areas for improvement’, leaving the apraisee exposed to capricious revisioning. On the issue of Quality, there are also questions of whether a manager’s expertise can always extend to appraising quality of research, particularly when the parameters for exercising judgment are far from clear.

‘Quantity’, similarly, is unrevealing. The presupposition is that more is better, so how does this sit with likely institutional policies on work-life balance and stress management? And what measures are being used? Hours logged in the classroom? Or evenings spent answering students’ questions on email. Or thank-you cards sent by finalists? Are teaching and research both considered ‘outputs’? We note an ironic choice of language in ‘quantity’, especially given the non-judgmental use of ‘engagement’ that universities usually apply to student work.

Job knowledge might be an issue for new academics in post, and so seems superfluous beyond the probation review. Digging a little deeper, though, many of us observe that ‘procedures’ are shifting and transitory under conditions of volatility in higher education regulation. Given the secrecy and lack of consultation with which those changes are imposed, and the general regulatory ‘churn’, to require familiarity, is like asking appraisees to apprehend a mirage.

Perhaps the most pernicious trait desired in this forlorn framework is ‘attitude’. By what ‘benchmark’ are we being evaluated? I would counter that there are occasions where a negative attitude is beneficial to the academic community. Are we required to embrace ineffectual managers, or unenthusiastic students? Should we tolerate abusive phone calls from parents with equanimity? My attitude is my attitude, thanks. I don’t need your evaluation. In any case, how do you suggest remedying ‘attitude’ without seriously compromising my initiative (another category), or academic standards? Never underestimate the positive power of negative thinking, is my motto.

Appearance is yet another superficial and subjective judgement. Here are my prejudices: no sweat pants, no logo t-shirts, no leggings, no flip-flops. But if you ask the students, they are generally very welcoming of lecturers who mirror the informality of their ‘customers’. Possibly, though, this dispensation applies much more to male lecturers than to women. So what is being evaluated here, apart from some manager’s notion of gender-normed corporate dress code conformity?

Attendance and punctuality suggest a rigid and compliant personality, which is undermined by the desired qualities of ‘initiative’ and ‘flexibility’. The latter, particularly, is likely to escape actual validation. Our flexibility makes itself known to family as work bleeds into leisure and domestic life. The continual peeking at email, the agenda planning that invades a run, the feelings of guilt at taking a whole Sunday off during the marking season. It makes itself known as we submit to ridiculous marking turnaround times in order to satisfy intensifying demands for more feedback. It shortens our careers.

This is the world of university performance management at its most unthinking. Cheeringly, it is beginning to be challenged even from within. According to their recent report, the Leadership Foundation for Higher Education would find this appraisal ‘not fit for purpose’. The report  distinguishes between stewardship and agency approaches to performance management, and urges universities to consider a more flexible application of these. Stewardship approaches “focus on long-term outcomes through people’s knowledge and values, autonomy and shared leadership within a high trust environment”. By contrast, “agency approaches focus on short-term results or outputs through greater monitoring and control”. I can probably guess which one seems more familiar to most academics, for whom autonomy, shared leadership and high trust working environments reside in the folklore of a previous generation.

However (to gleefully break a Govian rule), institutions with a mission that is focused on “long-term and highly complex goals, which are difficult or very costly to measure (e.g., research excellence, contribution to society)” are more likely to benefit from incorporating a stewardship approach to performance management. If we keep up the pressure, if we call out the pointless appeasement of what Power has described as ‘rituals of verification’ (1997:1), if we undermine the assumptions, then perhaps managers will start to align complexity of mission with a more delicate implementation of performance management.

References

Franco-Santos, M., Rivera, P. and Bourne, M. (2014), Performance Management in UK Higher Education Institutions, Leadership Foundation for Higher Education, London.

http://www.som.cranfield.ac.uk/som/dinamic-content/news/documents/PerformanceManagementinUKHigherEducationInstitutions.pdf

Power, Michael. 1997. The Audit Society:  Rituals of Verification. Oxford: Oxford University Press.