Objectivity of the peer‐review process: Enduring myth, reality, and possible remedies

Abstract

Key points The ‘replication crisis’ in science raises serious questions about the objectivity and reliability of the peer‐review process. Much of the literature, contributed on the topic in the past by former editors, has focused on the role of reviewers, and their possible biases. However, experience in practice shows that editors also contribute significantly, at different levels, to the lack of objectivity of peer‐review. Various techniques, including network analysis and machine learning, can be implemented to remedy the situation and restore the gatekeeping role of peer review.

Improving equity, diversity, and inclusion in academia

“…The academic publication process is built on objectivity [ 1 ], gender and socio-cultural neutrality [ 2 ], and respect for human and animal rights. Hence, equity, diversity, and inclusion (EDI) are essential in publication processes, among other academic spaces.…”

Finding our voice—Scholarship in peer review

“…While there is much debate over its objectivity, reliability and effectiveness, 1,2 peer review is a prevalent and integral part of scholarly publishing. Peer reviewers play essential roles in this process as independent assessors who evaluate and comment on manuscripts prior to publication in order to maintain academic standards of published articles.…”

What More Can We Do in Peer Review to Achieve Equity in Scientific Publication?

“…However, when referring to the comments of reviewers to make decisions on whether the manuscript can be published, editors lack the necessary professional judgment and sense of responsibility. 4 If editors receive a negative evaluation from a reviewer, they are more likely to make a decision that the manuscript can’t be published, 5 but rarely seriously evaluates whether the negative comments of the reviewer are reasonable.…”

“Data‐driven” versus “question‐driven” soil research

“…That does not mean that we should forgo entirely the use of data mining and machine learning algorithms. As with any tool, we should determine carefully whether they can help us achieve our goals, use them when we think they have potential (e.g., Baveye, 2021), and discard them promptly when we have compelling reasons to consider they do not (e.g., discussion about upscaling in Baveye et al, 2018).…”