Bibliometrics in Academic Recruitment: A Screening Tool Rather than a Game Changer

Abstract

This paper investigates the use of metrics to recruit professors for academic positions. We analyzed confidential reports with candidate evaluations in economics, sociology, physics, and informatics at the University of Oslo between 2000 and 2017. These unique data enabled us to explore how metrics were applied in these evaluations in relation to other assessment criteria. Despite being important evaluation criteria, metrics were seldom the most salient criteria in candidate evaluations. Moreover, metrics were applied chiefly as a screening tool to decrease the number of eligible candidates and not as a replacement for peer review. Contrary to the literature suggesting an escalation of metrics, we foremost detected stable assessment practices with only a modestly increased reliance on metrics. In addition, the use of metrics proved strongly dependent on disciplines where the disciplines applied metrics corresponding to their evaluation cultures. These robust evaluation practices provide an empirical example of how core university processes are chiefly characterized by path-dependency mechanisms, and only moderately by isomorphism. Additionally, the disciplinary-dependent spread of metrics offers a theoretical illustration of how travelling standards such as metrics are not only diffused but rather translated to fit the local context, resulting in heterogeneity and context-dependent spread.

Are evaluative cultures national or global? A cross-national study on evaluative cultures in academic recruitment processes in Europe

“…The second suggested that increasing internationalization of disciplines would lead to isomorphism and a prominence of field-dependent preferences (Lamont 2009). In line with the second expectation and based on prior studies, we expected that economists would more strongly value candidates' bibliometrics (Hylmö 2018), while physicists would place more value on their research profiles due to a larger degree of functional dependency among researchers (Reymert 2020;Whitley 2000). In line with our first expectation, we expected researchers in the Netherlands to be least concerned with bibliometrics because their funding model does not include bibliometric indicators (Aagaard 2015;Sivertsen 2017).…”

“…Therefore, we assume that both fields apply different evaluative criteria where researchers in economics should place stronger emphasis on the importance of metrics, while researchers in physics, to a larger extent, emphasize candidates' research profiles in order to assess how they might fit into a research group. This difference between the two fields has also been documented by recent qualitative studies building on confidential recruitment reports (Reymert 2020). Finally, both fields are characterized by a high degree of internationalization.…”

“…These results aligned with prior studies on academic recruitment processes, which have shown that economists emphasize publications in top journals (Hylmö 2018). The physicists put stronger emphasis on candidates' matching research profiles and language skills aligned with prior studies (Reymert 2020); moreover, their emphasis on general impression could be understood in terms of how they work in research groups where individual researchers have a specific role (Whitley 2000), thereby emphasizing a more pressing need to select candidates with compatible profiles and competencies. These field differences were similar across national borders, indicating that despite national differences in recruitment (Alfonso 2016), the evaluation processes are embedded in their fields' epistemic traditions (Lamont 2009), which are less sensitive to national considerations (Musselin 2010).…”

Kjønn og akademia

“…Samtidig tillegges publisering betydning også i andre sammenhenger innenfor akademia. Tall over vitenskapelig publisering -såkalte bibliometriske indikatorer -brukes i finansierings-og evalueringssammenheng (Wilsdon mfl., 2015), og antall publikasjoner har betydning for ansettelser og stillingsopprykk (Reymert, 2020). Internt i forskersamfunnet ser en også at mange markedsfører sine prestasjoner gjennom å vise til publiseringsmål som h-indeks eller RG-skår på Researchgate -en type «Facebook» for forskere.…”

Gender Equality in Academia: FROM KNOWLEDGE TO CHANGE

“…According to Campbell et al (2010), there seems to be a consensus among researchers that the study of publications and citations, bibliometry, is an "objective, reliable, and cost-effective measure of peer-reviewed research outputs" (Campbell et al, 2010, p. 66). Various indicators of scientific productivity are used as a basis for employment, promotion, and allocation of research funding (Reymert, 2020;Wilsdon et al, 2015). In addition, bibliometry is considered a reliable tool for assessing and managing R&D funding (European Science Foundation, 2009).…”