The Miracle of Peer Review and Development in Science: An Agent-Based Model

Abstract

It is not easy to rationalize how peer review, as the current grassroots of science, can work based on voluntary contributions of reviewers. There is no rationale to write impartial and thorough evaluations. If reviewers are unmotivated to carefully select high quality contributions, there is no risk in submitting low-quality work by authors. As a result, scientists face a social dilemma: if everyone acts according to his or her own selfinterest, the outcome is low scientific quality. We examine how the increased relevance of public good benefits (journal impact factor), the editorial policy of handling incoming reviews, and the acceptance decisions that take into account reputational information, can help the evolution of high-quality contributions from authors. High effort from the side of reviewers is problematic even if authors cooperate: reviewers are still best off by producing low-quality reviews, which does not hinder scientific development, just adds random noise and unnecessary costs to it. We show with agent-based simulations why certain selfemerged current practices, such as the increased reliance on journal metrics and the reputation bias in acceptance, work efficiently for scientific development. Our results find no proper guidelines, however, how the system of voluntary peer review with impartial and thorough evaluations could be sustainable jointly with rapid scientific development.

Evaluating alternative systems of peer review: a large-scale agent-based modelling approach to scientific publication

“…Author, editor and referee behaviour has been extensively studied with ABM and other modelling approaches. Some authors focused on how the number of reviewers, reciprocity, rationality and other motives between referees and authors affect the quality of peer review, and others redesigned models to replicate their results (Bianchi and Squazzoni 2015 ; Squazzoni and Gandelli 2013 ; Thurner and Hanel 2011 ; Paolucci and Grimaldo 2014 ; Righi and Takács 2017 ). Others modelled how objectivity and subjectivity in reviewers’ decisions macroscopically bias peer review (Park et al 2014 ) or estimated the level of bias necessary to affect peer review in grant applications (Day 2015).…”