The state of the art in peer review

Abstract

Scholarly communication is in a perpetual state of disruption. Within this, peer review of research articles remains an essential part of the formal publication process, distinguishing it from virtually all other modes of communication. In the last several years, there has been an explosive wave of innovation in peer review research, platforms, discussions, tools and services. This is largely coupled with the ongoing and parallel evolution of scholarly communication as it adapts to rapidly changing environments, within what is widely considered as the ‘open research’ or ‘open science’ movement. Here, we summarise the current ebb and flow around changes to peer review and consider its role in a modern digital research and communications infrastructure and suggest why uptake of new models of peer review appears to have been so low compared to what is often viewed as the ‘traditional’ method of peer review. Finally, we offer some insight into the potential futures of scholarly peer review and consider what impacts this might have on the broader scholarly research ecosystem. In particular, we focus on the key traits of certification and reputation, moderation and quality control and engagement incentives, and discuss how these interact with socio-technical aspects of peer review and academic culture.

A tale of two 'opens': intersections between Free and Open Source Software and Open Scholarship

“…Journal articles have only slightly evolved since their origins, ideally now containing sufficient documentation of research processes such that others can build upon or repeat that research. Despite the advent and enormous growth of Web-based technologies that permit more effective and low-cost methods of sharing and dissemination, greater epistemic diversity in scholarship, and increased democratisation of academic knowledge (Owen, Macnaghten, and Stilgoe 2012;Cowley 2015;Hartgerink and van Zelst 2018;Hartgerink 2019;Tennant 2018b;Martin 2019), scholarly journals and journal articles remain largely as isolated and low-tech digital versions of their printed analogues. Appearance of these new technologies catalysed the modern Open Access (OA) movement, which, similar to FOSS, primarily focuses on providing unrestricted access to research articles (Guédon 2008;Suber 2012;Tennant et al 2016) and helping to increase the global/public circulation of research information beyond professional scientists (Adcock and Fottrell 2008;Laakso et al 2011;Bacevic and Muellerleile 2018).…”

“…Here, commodification and privatisation of research has become the de facto norm for much of the industry, which relied on exclusionary business models to support itself. Because of this, some actors in the private sector, as well as a number of professional academic societies, have even taken active stances to subvert, co-opt, or even stop progress towards more inclusive and open systems (Posada and Chen 2018;Tennant 2018a). The hyper-competitive reward system only seems to have exacerbated this, instead promoting secrecy and individualism rather than any form of collective collaboration for knowledge generation (Merton 1968).…”

“…Prestige and reputation thus would return to principles of "good" scholarship, and not be outsourced to inappropriate and proprietary measures. Elements of such a system can already be seen in a range of digital platforms that leverage the power of networked communities to gain a deeper understanding of people and processes (Tennant 2018b). The bottom line is that the right measures of reputation across all levels of Open Scholarship could incentivize the right motivation.…”

“…Scholarly communities should be aware of the social contexts surrounding knowledge transfer(Gibbons 1999;Morgan et al 2018), and use this to create the best cultural environment and shared values possible to attract these motivations of scholars. Given some documented effects of openness such as wider readership and access(Levin 2015), boosting collaboration and transparency(Peters and Roberts 2015), democratisation of scienceTennant 2018a) and promoting accountability and responsible research(Resnik 2006), ideally all stakeholders involved in the process of scholarly communication can and should be supportive of this trend, both in principle and in actions. However, that is not always the case.…”

Ten myths around open scholarly publishing

“…But these subtleties are lost on the general public, who often only hear the myth that published in a journal with peer review is the "gold standard" and can erroneously equate published research with the truth. Thus, more care must be taken over how peer review, and the results of peer reviewed research, are communicated to non-specialist audiences; particularly during a time in which a range of technical changes and a deeper appreciation of the complexities of peer review are emerging (Bravo et al 2019;Tennant 2018;Squazzoni, Grimaldo, and Marušić 2017;Allen et al 2018). This will be needed as the scholarly publishing system has to confront wider issues such as retractions (Budd, Sievert, and Schultz 1998b;Fang and Casadevall 2011;Moylan and Kowalczuk 2016) and replication or reproducibility 'crisis' (Collaboration 2015;Munafò et al 2017a;Fanelli 2018).…”