Bias and effort in peer review

Abstract

Here, we develop a theory of the relationship between the reviewer's effort and bias in peer review. From this theory, it follows that journal editors might employ biased reviewers because they shirk less. This creates an incentive for the editor to use monitoring mechanisms (e.g., associate editors supervising the peer review process) that mitigate the resulting bias in the reviewers' recommendations. The supervision of associate editors could encourage journal editors to employ more extreme reviewers. This theory helps to explain the presence of bias in peer review. To mitigate shirking by a reviewer, the journal editor may assign biased referees to generate information about the manuscript's quality and subject the reviewer's recommendations to supervision by a more aligned associate editor. IntroductionPeer review of articles that appear in academic journals is required by the scientific community, and articles are supposed to be selected for publication on the basis of merit and appropriateness alone. In some journals, after the manuscript has been received and deemed appropriate with respect to submission criteria, the journal editor assigns the manuscript to an associate editor. In such a case, the associate editor is usually responsible for the management of the peer review of manuscripts by members of the relevant research field. In other cases, academic journals are run by only one editor (unassisted, or with a managing editor who does not make editorial decisions). In some scholarly journals, rather than a single editor with several associate editors, multiple independent editors operate the journal, or there are editorial boards that decide about submissions. Nevertheless, the quality level of the journal is increased by assuring the selection of appropriate referees to identify high-quality articles and by efficiently managing the peer review process, (Burnham, 1990;Campanario, 1998aCampanario, , 1998b.Reviewers are identified through peer contact or references listed at the end of the manuscript. At some academic journals, authors are invited to nominate a minimum number of reviewers. Occasionally this is required by the submission system. Then, the associate editor will determine the disposition of the manuscript, based on the remarks of the reviewers, and his own assessment of the manuscript. The decision for a major or minor revision will be conveyed to the author by the associate editor. In general, novelty, quality of technical content, and appropriateness for publication are the criteria for the manuscript to be accepted. The goal of the peer review process is to seek fundamental knowledge and identify the most promising research. In many ideal depictions, peer review processes are understood as providing a "system of institutionalized vigilance in the self-regulation of knowledge communities" (Merton, 1973, p. 339), and thus "peer expertise is coordinated to vet the quality and feasibility of submitted work" (Lee, Sugimoto, Zhang, & Cronin, 2013, p. 3). A description of the major findings in...

Problematizing Peer Review: Academic Librarians’ Pedagogical Approaches to Peer Review

“…Library literature discusses peer review under the auspices of open access publishing (especially notable is Emily Ford's (2013Ford's ( , 2016aFord's ( , 2016bFord's ( , 2018 work on open peer review), as a peer-to-peer editing activity for improving student writing (Wheeler, 2011;Zwicky & Hands, 2016), or in terms of information evaluation (Rosenzweig, Thill, & Lambert, 2019;Shotton, 2012;Warren & Duckett, 2010), but rarely in terms of critically analyzing the process itself (Potvin, 2017). Articles discussing the value and role of peer review in academic publishing (Kaspar, 2016;Weller, 1995) are most likely to address critiques of the peer review process, while there are a growing number that directly confront bias in the peer review process (García, Rodriguez-Sánchez, & Fdez-Valdivia, 2015) and the moments in which peer review fails at its own stated goals (Leon, 2014).…”

Laughter through tears: Unprofessional review comments as humor on the ShitMyReviewersSay Twitter account

“…Extensive research has been done outside the field of linguistics on a whole gamut of factors contributing to the different effects of single-blind and double-blind reviewing (see Tomkins et al 2017), as well as various inequalities and biases in the review process (see e.g., Garcia et al 2015;Lee et al 2013). For instance, it has been shownquite unsurprisinglythat in the single-blind review process, referees favor papers from recognized authors, as well as top universities and companies (Tomkins et al 2017).…”

Peer Review in Controversial Topics—A Case Study of 9/11

“…One method is to select reviewers who are more biased or even hostile to a paper on the assumption that such reviewers are more likely to search for flaws in the paper. The negative effects of the reviewers' biases would then be offset by an associate editor who evaluates the alignment quality of review reports [55]. In this writer's opinion, this is a dubious procedure at best and one that is both unpredictable and likely to backfire.…”