Media Release
7January, 2009

Research system ‘stifling quality’

A study of New Zealand economists suggests that the flagship university research quality assessment system isn’t yet bringing the expected increase in research quality, and may be weakening incentives in the academic labour market for economists to publish higher quality articles.

Professor John Gibson and Professor John Tressler at the University of Waikato Management School, and Professor David Anderson of Queen’s University in Ontario, Canada, compared the ranks and publication records of all New Zealand-based academic economists to see how the Performance Based Research Fund (PBRF) had changed the relationship between research productivity and academic rank. The Tertiary Education Commission funds universities on the basis of their performance.

The first data set was from 1999 before the introduction of the PBRF; the second data set was from 2007 after two of the PBRF research quality assessments, in 2003 and in 2006.

“Assessments like the PBRF only make economic sense if the academics respond to the market signals,” says Prof Gibson. “The aim of the PBRF is to raise research quality, so we would expect it to cause academics to concentrate more on quality publications and that should show up in a stronger link between research quality and rank, especially because rank is a good proxy for pay at New Zealand universities.”

In fact, says Prof Gibson, contrary to what might be expected from the PBRF, the relative returns to quality seem to have gone down rather than up since its introduction.

“While the total volume of research output increased by one-third, average quality declined according to three of our five quality measures. Overall, the returns to quantity relative to quality increases from 1999 to 2007, particularly in the case of new appointments.

“Our findings suggest that university economists are more likely to gain promotion by increasing their research output rather than raising the quality of their work.”

Prof Gibson says if the same findings hold true for other academic disciplines, this will have serious implications for the PBRF. “If individual academics find they gain no direct benefit from improving the quality of their research, then we cannot assume the gains from the PBRF will outweigh the very high direct and indirect costs of such an assessment.”

ENDS