Desingularization and Dequalification: A Foray Into Ranking Production and Utilization Processes

Abstract

Although some authors highlight the benefits of journal rankings, previous research is often highly critical of them, insinuating that they can lead to desingularization of academic journals (i.e., their impoverishment and standardization) and dequalification of researchers (i.e., a weakening of researchers' ability to evaluate academic research). However, as very few authors have empirically assessed these presumptions, we aim to address this gap in the literature. Based on Lucien Karpik's notions of singularities, judgment devices, forms of involvement, and emulation and rivalry, we assess whether the processes surrounding the production and use of journal rankings might lead to desingularization and dequalification. Our findings support previous research by highlighting that processes where passivity and heteronomy (i.e., lack of autonomy) prevail, are conducive to desingularization, rivalry and dequalification. Our findings, however, introduce some nuances into the debate by underscoring instances where emulation logic is employed instead of mere rivalry logic, and where substantial judgment devices and active involvement are mobilized in the production and use of rankings, thereby somewhat alleviating the spread of desingularization and dequalification. Ultimately, our study raises questions that point to a need for serious collective reflection within the academic community on the processes by which published research is evaluated.

Accounting as a dichotomised discipline: An analysis of the source materials used in the construction of accounting articles

“…Thus, in some cases at least, journal rankings are replacing the need to think critically about the strengths and weaknesses of a particular reference source -the fact that it appears in a high-ranked journal is viewed as sufficient justification for its usage. Picard et al (2018) provides an insight into this phenomenon. They apply Karpik's analysis of the economics of singularities (2007,2010) to the case of academic articles (Karpik, 2011).…”

“…The most notable trends in reference sources are the rise in the usage of high-ranked references sources and the journal-wide decline in references from books and monographs. Given the growing tendency for assessors and reviewers to utilise journal rankings as proxies for an article's 'quality' (Walker et al, 2018;Picard et al, 2018), it is possible that authors will seek to signal the quality of their paper to editors and reviewers by drawing heavily upon high-ranked source materials. This is a phenomenon driven by the rise of journal ranking lists, which we discussed earlier in this paper.…”

Information Retrieval and Knowledge Organization: A Perspective from the Philosophy of Science

“…This is, however, an open hypothesis, which seems to have been challenged by [39], who found that important papers are more and more published in non-elite journals. For a criticism of the journal impact factor, see, for example, [40]). (3) selecting documents based on principles used in so-called evidence-based research (e.g., studies based on double blind clinical trials); In evidence-based medicine (or evidence-based practice in general, EBP) the trustworthiness of claims about the effectiveness of a given treatment are classified according to the quality of the research methods employed.…”

Academic engagement in policy-making and social and environmental reporting

“…Finally, academics lack incentives to allocate their time and effort to engage in policy-making and produce relevant research for regulators (Singleton-Green, 2010) instead to dedicating their time to research (Fülbier et al , 2009). This impediment is especially acute for early-stage researchers, affected by the short-termism of the publication requirements to thrive in their academic careers (Picard et al , 2018). Fülbier et al (2009) suggest that producing relevant research for policy-making should also be considered when evaluating academic production; however, concerns are expressed about the possible negative consequences of adding engagement in the management of the academic performance (Bebbington et al , 2017).…”

“…We wonder whether the stupidity introduced in current research assessment exercises in which the young generation of scholars is being nurtured will create such a long-term temperament and collective dynamics required for a new epistemic community to emerge. The reign of journal rankings in Spain (as in other countries) and demands for short-term academic performance are fostering heteronomy (Picard et al , 2018), rather than the autonomy required for young scholars to develop long-term research projects, as well as a broad understanding of substantive issues, such as the role of academics and the public interest.…”