Incentives for Research Effort: An Evolutionary Model of Publication Markets with Double-Blind and Open Review

Abstract

Contemporary debates about scientific institutions and practice feature many proposed reforms. Most of these require increased efforts from scientists. But how do scientists’ incentives for effort interact? How can scientific institutions encourage scientists to invest effort in research? We explore these questions using a game-theoretic model of publication markets. We employ a base game between authors and reviewers, before assessing some of its tendencies by means of analysis and simulations. We compare how the effort expenditures of these groups interact in our model under a variety of settings, such as double-blind and open review systems. We make a number of findings, including that open review can increase the effort of authors in a range of circumstances and that these effects can manifest in a policy-relevant period of time. However, we find that open review’s impact on authors’ efforts is sensitive to the strength of several other influences.

On the peer review reports: does size matter?

“…In other words, we only put on Publons well-produced reports. A recent paper (Radzvilas et al, 2022), based on evolutionary game theory, indicates that the type of peer review of the journal directly affects the behavior of researchers and reviewers. Specifically, the paper indicates that if the journal practices open peer review, it encourages researchers and reviewers to put in more effort (because it directly affects their reputation).…”

Prevalence of Short Peer Reviews in 3 Leading General Medical Journals

“…Additionally, based on information provided by each of the journals on their websites, it appears that The BMJ provides the most incentives (all nonmonetary) for reviewers (eg, article processing charge discounts, continuing medical education credits, and certificates) . It is, thus, possible that increasing accountability and providing incentives are approaches that could encourage reviewers to provide more detailed reviews and that similar measures could also increase review length at other journals such as BMC Medicine and PLOS Medicine . Norm-based interventions have a strong evidence base for changing behaviors and could also potentially be used in this context .…”

“…[27][28][29] It is, thus, possible that increasing accountability and providing incentives are approaches that could encourage reviewers to provide more detailed reviews and that similar measures could also increase review length at other journals such as BMC Medicine and PLOS Medicine. 30,31 Norm-based interventions have a strong evidence base for changing behaviors and could also potentially be used in this context. 32 Future studies could investigate the impact of interventions targeting norms on peer review word count, such as posting sample reviews and indicating to reviewers how their review length compares with other reviews for similar articles.…”

The Role of Publons in the Context of Open Peer Review

“…However, peer review standards in a wide range of journals and publishers still varies widely [ 91 ], even if many of them fall under the same umbrellas of ethics-promoting organizations such as COPE or the ICMJE. There is also doubt whether purely reputational gains will be a sustainable model for OPR to become the main or predominant peer review model because there are many aspects that may hinder the incentivization of academics to openly participate in such schemes [ 92 ]. We believe that the sudden disappearance of all Publons-related websites, even if the information was integrated into WoS, may demotivate some academics that may have invested heavily (personally and professionally) into the Publons brand.…”