Black Box, Pandora's Box or Virtual Toolbox? An Experiment in a Journal's Transparent Peer Review on the Web

Maharg, Paul; Duncan, Nigel

doi:10.1080/13600860701492104

“…On the other hand, eponymous peer review has the potential to inject responsibility into the system by encouraging increased civility, accountability, declaration of biases and conflicts of interest, and more thoughtful reviews ( Boldt, 2011 ; Cope & Kalantzis, 2009 ; Fitzpatrick, 2010 ; Janowicz & Hitzler, 2012 ; Lipworth et al , 2011 ; Mulligan et al , 2013 ). Identification also helps to extend the process to become more of an ongoing, community-driven dialogue rather than a singular, static event ( Bornmann et al , 2012 ; Maharg & Duncan, 2007 ). However, there is scope for the peer review to become less critical, skewed, and biased by community selectivity.…”

Section: The Traits and Trends Affecting Modern Peer Reviewmentioning

confidence: 99%

A multi-disciplinary perspective on emergent and future innovations in peer review

Tennant

¹

,

Dugan

²

,

Graziotin

³

et al. 2017

View full text Add to dashboard Cite

Peer review of research articles is a core part of our scholarly communication system. In spite of its importance, the status and purpose of peer review is often contested. What is its role in our modern digital research and communications infrastructure? Does it perform to the high standards with which it is generally regarded? Studies of peer review have shown that it is prone to bias and abuse in numerous dimensions, frequently unreliable, and can fail to detect even fraudulent research. With the advent of web technologies, we are now witnessing a phase of innovation and experimentation in our approaches to peer review. These developments prompted us to examine emerging models of peer review from a range of disciplines and venues, and to ask how they might address some of the issues with our current systems of peer review. We examine the functionality of a range of social Web platforms, and compare these with the traits underlying a viable peer review system: quality control, quantified performance metrics as engagement incentives, and certification and reputation. Ideally, any new systems will demonstrate that they out-perform and reduce the biases of existing models as much as possible. We conclude that there is considerable scope for new peer review initiatives to be developed, each with their own potential issues and advantages. We also propose a novel hybrid platform model that could, at least partially, resolve many of the socio-technical issues associated with peer review, and potentially disrupt the entire scholarly communication system. Success for any such development relies on reaching a critical threshold of research community engagement with both the process and the platform, and therefore cannot be achieved without a significant change of incentives in research environments.

show abstract

“…Identification also helps to extend the process to become more of an ongoing, community-driven dialogue rather than a singular, static event ( Bornmann et al , 2012; Maharg & Duncan, 2007). However, there is scope for the peer review to become less critical, skewed, and biased by community selectivity.…”

Section: The Traits and Trends Affecting Modern Peer Reviewmentioning

confidence: 99%

A multi-disciplinary perspective on emergent and future innovations in peer review

Dugan

¹

,

Graziotin

²

,

Jacques

³

et al. 2017

View full text Add to dashboard Cite

Peer review of research articles is a core part of our scholarly communication system. In spite of its importance, the status and purpose of peer review is often contested. What is its role in our modern digital research and communications infrastructure? Does it perform to the high standards with which it is generally regarded? Studies of peer review have shown that it is prone to bias and abuse in numerous dimensions, frequently unreliable, and can fail to detect even fraudulent research. With the advent of web technologies, we are now witnessing a phase of innovation and experimentation in our approaches to peer review. These developments prompted us to examine emerging models of peer review from a range of disciplines and venues, and to ask how they might address some of the issues with our current systems of peer review. We examine the functionality of a range of social Web platforms, and compare these with the traits underlying a viable peer review system: quality control, quantified performance metrics as engagement incentives, and certification and reputation. Ideally, any new systems will demonstrate that they out-perform and reduce the biases of existing models as much as possible. We conclude that there is considerable scope for new peer review initiatives to be developed, each with their own potential issues and advantages. We also propose a novel hybrid platform model that could, at least partially, resolve many of the socio-technical issues associated with peer review, and potentially disrupt the entire scholarly communication system. Success for any such development relies on reaching a critical threshold of research community engagement with both the process and the platform, and therefore cannot be achieved without a significant change of incentives in research environments.

show abstract

“…More broadly speaking, OPR provides the scholarly community an insight into author/referee conversations during the review process. Surfacing these conversations provides readers an expanded contextual discussion of the subject at hand, and enriches science communication for all stakeholders ( Lipworth & Kerridge, 2011; Fitzpatrick, 2010; Friedman et al , 2010; Maharg & Duncan, 2007). Finally, perhaps the most convincing pro argument for OPR asserts that OPR processes allow for quicker publication and dissemination of scientific findings ( Hu et al , 2010; Cope & Kalantzis, 2009; Pöschl, 2004).…”

Section: Why Open Peer Review?mentioning

confidence: 99%

Open peer review at four STEM journals: an observational overview

Ford

¹

2015

View full text Add to dashboard Cite

Open peer review, peer review where authors' and reviewers' identities are disclosed to one another, is a growing trend in scholarly publishing. Through observation of four journals in STEM disciplines, PLoS One, Atmospheric Chemistry & Physics, PeerJ, and F1000Research, an observational overview is conducted. The overview relies on defined characteristics of open peer review. Results show that despite differing open peer review implementations, each journal retains editorial involvement in scholarly publishing. Further, the analysis shows that only one of these implementations is fully transparent in its peer review and decision making process. Finally, the overview contends that journals should clearly outline peer review and editorial processes in order to allow for open peer review to be better understood and adopted by authors, reviewers, editors, and readers of science communications.

show abstract

Black Box, Pandora's Box or Virtual Toolbox? An Experiment in a Journal's Transparent Peer Review on the Web

Abstract: This is the accepted version of the paper.This version of the publication may differ from the final published version. Permanent repository link:

Cited by 10 publications

References 2 publications

A multi-disciplinary perspective on emergent and future innovations in peer review

A multi-disciplinary perspective on emergent and future innovations in peer review

A multi-disciplinary perspective on emergent and future innovations in peer review

Open peer review at four STEM journals: an observational overview

Contact Info

Product

Resources

About