Tables 'affect university policy'

The influence of tables is expected to continue growing

Universities in England are "strongly influenced" by league tables despite concerns about the way they are compiled, a study has found.

Three newspaper tables and two world rankings were analysed for the Higher Education Funding Council for England.

The researchers said efforts to move up the tables might clash with policy objectives and academic standards.

University leaders say there is still insufficient transparency about the way the league tables are compiled.

The study was undertaken by the Centre for Higher Education Research and Information at the Open University and Hobsons Research.

They looked at the Sunday Times Good University Guide, the Times Good University Guide, the Guardian University Guide, the Academic Ranking of World Universities published by Shanghai Jiao Tong University Institute of Higher Education and the Times Higher Education Supplement (now THE)/Quacquarelli Symonds ranking.

Reputation

Their report, Counting what is measured or measuring what counts?, published by Hefce during its annual conference, said there was clearly a demand for league tables among prospective students and others.

It said the rankings largely reflected "reputational" factors such as students' qualifications on entry, the level of degree they attained and Research Assessment Exercise grades.

The tables measured the data available, some of which were poor indicators for the qualities they were supposed to reflect, such as "excellence" or "a world class university".

"Also the weightings applied do not always seem to have the desired effect on the overall scores for institutions.

"This brings into question the validity of the overall tables," the report said.

The way scores were worked out was not always transparent.

"The resulting rankings largely reflect reputational factors and not necessarily the quality or performance of institutions."

Staff morale

Yet institutions were strongly influenced by them.

"League tables are being used by many institutions as key performance indicators and, in some cases, strategic targets. They are being used by some senior management teams and governing bodies as one of several drivers for internal change."

Despite widespread scepticism, staff morale was affected.

The report concludes: "The influence of league tables is increasing both nationally and internationally, and cannot be ignored despite serious methodological limitations.

"They are being used for a broader range of purposes than originally intended, and being bestowed with more meaning than the data alone may bear.

"It is possible that the influence of league tables will increase further if the cap on tuition fees for full-time undergraduate courses is raised or lifted altogether.

"It is possible that ranking position will affect an institution's ability to charge the highest fees across all its courses."

'Misunderstandings'

Hefce chief executive Prof David Eastwood said: "Our purpose in commissioning this research was not to create an authorised 'official' league table or to endorse any particular approach, but to stimulate informed debate about league tables across the higher education sector.

"We hope that this will lead to improvements in league table methodologies and enable users to understand their complexities better and avoid misunderstandings."

The chief executive of vice-chancellors' group Universities UK, Diana Warwick, said it had been working with league table compilers to encourage greater transparency about their methodology and measures, including those that demonstrated the "value added" by institutions.

"We also note the finding that league table measures are poor proxies for the qualities the compilers are aiming to identify, and that league table formats could be updated to make them more interactive.

"We hope the compilers take these findings into account in future league tables."

National Union of Students vice-president Wes Streeting said the study proved what the union had long argued: "League tables can only advise prospective students on the prestige associated with a university, as opposed to how well it might suit their own individual needs.

"Current league tables disproportionately favour research-intensive institutions - this is unhelpful to most prospective undergraduates, who are not going to be taking research degrees."

E-mail this to a friend

Printable version