The satisfaction trap

12 October 2026

Senior Consultant, Elena Liquete asks whether universities are measuring the right things, and what happens when the answer is no. It is a question with direct implications for how institutions position themselves and what they promise prospective students.

Lincoln Bishop University was, by one measure, the best university in Britain. In the National Student Survey, it came top for student experience year after year. Students loved it. Yet it's deficit trebled over three years and ultimately, it was taken over.

That got me thinking. Universities across the country chase NSS scores with the dedication they once reserved for research rankings. Lincoln Bishop's financial troubles while sitting at the top of that table is not a curiosity. It is a question about what we are actually measuring.

The NSS asks students whether they are satisfied with their teaching, their feedback, their academic support, and the organisation of their course. It asks whether they feel their voice is heard. Since 2023, it has asked about mental wellbeing services. What it does not ask is whether students learned anything difficult, whether they were pushed beyond what they thought they could do, or whether the experience will matter to them in years to come. Those questions are absent. Not because no one thought of them, but because those things are hard to measure. Satisfaction is easier to capture than growth, so satisfaction is what we measure.

The result is a metric that rewards universities for smoothing out friction. And friction, it turns out, matters rather a lot to learning.

Elizabeth and Robert Bjork at UCLA have spent decades documenting what they call "desirable difficulties" - conditions that make learning feel harder in the short term and stick better over time: Returning to material after gaps, mixing up problem types, withholding easy answers - these methods rarely feel good while you are doing them. Students in courses that use these methods often rate the teaching lower, not higher. The effort required is mistaken for poor instruction.

The most famous demonstration of this is more than fifty years old, which is itself part of the argument. Donald Naftulin, John Ware and Frank Donnelly hired a professional actor, gave him a fictional identity - "Dr Myron Fox, authority on the application of mathematics to human behaviour" - and set him loose on a group of psychiatrists and social workers. Dr Fox delivered a lecture of deliberate nonsense, contradicting himself, inserting jokes, and saying very little of substance. His audience rated the session as stimulating, thought-provoking and well organised. The researchers called it the Dr Fox effect. Student satisfaction, it turned out, was a measure of performance. Not pedagogy.

This is not an argument against listening to students. It is an argument about the difference between the moment of experience and its consequences. The teacher who pushes you hardest rarely wins the end-of-term survey. The seminar that challenges your assumptions is not comfortable while it is happening. I have never spoken to a graduate who named their most comfortable course as the one that changed them. The NSS is administered before most students have any professional context in which to evaluate what their degree actually gave them. It captures the moment, not the consequence.

Philip Stark and Richard Freishtat, in their 2014 analysis of student evaluations at Berkeley, found that scores correlate more strongly with course difficulty - easier courses get higher ratings - than with any identifiable measure of learning. They also found significant gender and racial bias in how individual instructors were rated. The metric was not measuring what it claimed to. It was measuring ease, warmth, and social comfort.

Universities know this. They just don't act on it. NSS scores have become reputational currency - league tables use them, marketing departments cite them, vice-chancellors track them. The result is a pressure toward exactly the wrong things: more support, less challenge; more feedback on drafts, fewer consequences for intellectual risk-taking; a university experience that feels, from the inside, very good and demands very little.

Lincoln Bishop was excellent at this. By every account it was a genuinely well-run institution that cared about its students. It is quite possible that its high scores reflected real investment in a supportive environment. It is also true that this investment cost money, that student satisfaction did not translate into the enrolment numbers required to sustain it, and that a university can score top in Britain and still find itself sold to a private company.

The system is not unhealthy because Lincoln Bishop failed. It is unhealthy because we measure the wrong things and acted surprised at the result. Student satisfaction is a reasonable proxy for student welfare. It is a poor proxy for educational quality, labour market preparation, or institutional sustainability. Conflating the three has produced a generation of universities skilled at gaming the survey and increasingly uncertain about everything else.

What would a better measurement look like? Graduate outcomes data exists and is published, though it lags by years and is easily gamed. Long-term earnings data is more honest but politically sensitive, and it flatters the prestigious rather than the effective.  It also penalises universities that send large numbers of graduates into teaching, nursing and other professions society depends on but pays modestly. A modest salary is not evidence of a poor education. Often it is evidence of a vocation.

The obvious alternative is to ask graduates themselves, some years on, what their degree gave them. The problem is attribution. By then many will have taken a Masters, trained with an employer, completed mandatory CPD, or simply gained experience. Separating what the undergraduate degree contributed from everything that came after is close to impossible. Timing has no good answer either. Ask too early and graduates cannot yet judge. Ask too late and the degree is one influence among many.  None of these are clean metrics.

But the first step is to stop treating the NSS as the answer to a question it was never designed to address. Students who are happy with their university are not necessarily students who have been educated. The teacher students remember is rarely the kindest one. The courses that change you are rarely the ones you would have chosen.

We should be capable of holding both truths at once: that student welfare matters, and that satisfaction is not the same as learning. Until we can, we will keep producing league tables that measure comfort and wondering why the top performers keep running out of money.

For universities, this is also a brand problem. When an institution builds its reputation on satisfaction scores, it is making a promise it cannot fully keep, and communicating a set of values that may not survive contact with what students actually need.  Deciding what your institution is actually for, and saying it plainly, is harder than chasing a survey score. It outlasts it, too.