The darker side of quantitative academic performance metrics

Abstract

Scientific information can be considered an economic commodity. 1 Authors produce papers as currency to acquire research employment, and, in turn, publishers sell these papers for profit. The appropriateness of monetising knowledge, and the sustainability of such business models, remains contentious. 2-4 But I do not wish to embroider on science economics per se. Instead I highlight here how current publishing behaviour has been influenced by economic incentives -behaviour that can now be predicted by basic economic models. 5 Understanding such behavioural changes in science culture will become helpful to identify potentially unethical publishing practices. 6,7 The use of quantitative academic performance metrics Eligibility criteria to secure employment or grants are generally based on quantitative academic performance indices, for example, number of papers, the h-index 8,9 , or citations per paper 10 . Science metrics have greatly advanced our understanding of publishing trends among authors, but are clearly also not without critique. 11 I have previously commented on the pitfalls of unsophisticated performance measures in evaluating academic success among scholars from developed and developing nations. 12 Current literature is filled with similar polemical opinions, where even choosing a journal to publish in has become an awkward affair. 13 Yet for administrators these records remain helpful to initially separate the wheat from the chaff, and are easy and free to obtain online. 14 Despite progress on reforming performance evaluations, such as the Leiden Manifesto 15 , it will take some time for the playing field to level out. The scene is therefore set for a new science culture of winners and losers. Aspiring academics will thus most likely have to actively manage their research productivity and performance 5 , more aggressively so than ever before in this hypercompetitive academic environment 6,16 .

Open science saves lives: lessons from the COVID-19 pandemic

“…Finally, we cannot exclude that some of the misuses and abuses that we have highlighted are a direct result of the current metric-centered evaluation of research and researchers which has already been shown to lead to questionable research practices in the past and has been the subject of criticism from scientists for decades [43,122,123]. Researchers have argued that the adoption of transparency should be coupled with the adoption of a more diverse set of metrics to evaluate researchers [124,125] or a rejection of metrics altogether [126,127] to truly limit questionable research practices. A wider adoption of these Open Science Principles cannot be achieved without the endorsement and support of institutions, publishers and funding bodies.…”

Open Science Saves Lives: Lessons from the COVID-19 Pandemic

“…Finally, we cannot exclude that some of the misuses and abuses that we have highlighted are a direct result of the current metric-centered evaluation of research and researchers which has already been shown to lead to questionable research practices in the past and has been the subject of criticism from scientists for decades [42, 126, 127]. Researchers have argued that the adoption of transparency should be coupled with the adoption of a more diverse set of metrics to evaluate researchers [128, 129] or a rejection of metrics altogether [130, 131] to truly limit questionable research practices. A wider adoption of these Open Science Principles cannot be achieved without the endorsement and support of institutions, publishers and funding bodies.…”

Is biomedical research self-correcting? Modelling insights on the persistence of spurious science

“…Science is from a philosophical standpoint self-correcting, and journals should revise policies which inadvertently impede this. Academic culture itself shares in the blame, as misguided efforts to gauge a researcher's worth by publication metrics lend themselves not to better science but to questionable research practices beyond the scope of this paper [50][51][52]. royalsocietypublishing.org/journal/rsos R. Soc.…”