Boon, bias or bane? The potential influence of reviewer recommendations on editorial decision-making

Penders, Bart; Ross-Hellauer, Tony; Marušić, Ana; Squazzoni, Flaminio; Mackay, Anson W.; Madan, Christopher R.; Shaw, David; Alam, Sabina; Mehmani, Bahar; Graziotin, Daniel; Nicholas, D. Chase

doi:10.20316/ese.2019.45.18013

Cited by 3 publications

(3 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Journals can also share data on their peer review workflows, including referee recommendations where possible [59]. The relationship of such recommendations to editorial decisions has currently only been performed at a relatively small scale for single journals [60,61] and requires further investigation [62]. Disclosure of this information would provide not only great insight into editorial decisions and their legitimacy, but also be useful in improving review and editorial management systems, including based around training and support [6].…”

Section: Roles Of Editors In Peer Reviewmentioning

confidence: 99%

The limitations to our understanding of peer review

Tennant¹,

Ross-Hellauer

2020

Res Integr Peer Rev

199

147

View full text Add to dashboard Cite

Peer review is embedded in the core of our knowledge generation systems, perceived as a method for establishing quality or scholarly legitimacy for research, while also often distributing academic prestige and standing on individuals. Despite its critical importance, it curiously remains poorly understood in a number of dimensions. In order to address this, we have analysed peer review to assess where the major gaps in our theoretical and empirical understanding of it lie. We identify core themes including editorial responsibility, the subjectivity and bias of reviewers, the function and quality of peer review, and the social and epistemic implications of peer review. The high-priority gaps are focused around increased accountability and justification in decision-making processes for editors and developing a deeper, empirical understanding of the social impact of peer review. Addressing this at the bare minimum will require the design of a consensus for a minimal set of standards for what constitutes peer review, and the development of a shared data infrastructure to support this. Such a field requires sustained funding and commitment from publishers and research funders, who both have a commitment to uphold the integrity of the published scholarly record. We use this to present a guide for the future of peer review, and the development of a new research discipline based on the study of peer review.

show abstract

Section: Roles Of Editors In Peer Reviewmentioning

confidence: 99%

The limitations to our understanding of peer review

Tennant¹,

Ross-Hellauer

2020

Res Integr Peer Rev

199

147

View full text Add to dashboard Cite

show abstract

“…Intervention studies via randomised controlled trials could also contribute to this. • Furthermore, regarding Open Identities, current models and evaluations of them hitherto heavily focus on the identities of some actors -particularly authors and reviewers -while still obfuscating the identities and roles of other actors in the editorial process, including handling editors, managing editors, and other editorial staff (Tennant et al 2019). To fully reach transparency in editorial review, the roles of these actors should be better understood.…”

Section: Evidence Gapsmentioning

confidence: 99%

‘Conditional Acceptance’ (additional experiments required): A scoping review of recent evidence on key aspects of Open Peer Review

Ross-Hellauer¹,

Horbach²

2022

Preprint

View full text Add to dashboard Cite

Diverse efforts are underway to reform the journal peer review system. Combined with growing interest in Open Science practices, Open Peer Review (OPR) has become of central concern to the scholarly community. However, what Open Peer Review is understood to encompass and how effective some of its elements are in meeting the expectations of the peer review system, are uncertain. This scoping review updates previous efforts to summarise research on OPR to date. Following the PRISMA methodological framework, it addresses the question: “What evidence has been reported in the scientific literature from 2017 to date regarding uptake, attitudes, and efficacy of two key aspects of Open Peer Review (Open Identities and Open Reports)?” The review identifies, analyses and synthesises 52 studies matching inclusion criteria, finding that OPR is growing, but still far from common practice. Our findings indicate positive attitudes towards Open Reports and more sceptical approaches to Open Identities. Changes in reviewer behaviour seem limited. and no evidence for lower acceptance rates of review invitations or slower turnaround times is reported. Concerns about power dynamics and potential backfiring on critical reviews are in need of further experimentation. We conclude that elements of OPR seem to be gaining acceptance, but more experimentation is needed. Evidence still mainly consists of either survey data or case studies of individual or few journals, not allowing for generalisability across fields and journals, and revealing no studies which compare the quality of review under Open Identities or Open Reports versus other modes of peer review.

show abstract

“…In this study, we have examined a recent bibliography regarding the pros and cons of the peer-review part of a system and its limitations [7], [8]. Although peer-review is considered highly important in article assessment [9], many problems occur in terms of biases from reviewers and editors alike [10], [11], [12]. Researchers argue that a great deal must change in order to enhance current procedures as mentioned in [13], [14].…”

Section: Related Workmentioning

confidence: 99%

A Blockchain Cloud Computing Middleware for Academic Manuscript Submission

Gazis

Anagnostakis

Kourmpetis

et al. 2022

Wseas Transactions on Business and Economics

View full text Add to dashboard Cite

One of the most important tasks in scientific publishing is the articles’ evaluation via the editorial board and the reviewers’ community. Additionally, in scientific publishing great concern exists regarding the peer-review process and how it can be further optimised to decrease the time from submission to the first decision, as well as increase the objectivity of the reviewers’ remarks ensuring that no bias or human error exists in the reviewing process. In order to address this issue, our article suggests a novice cloud framework for manuscript submission based on blockchain technology that further enhances the anonymity between authors and reviewers alike. Our method covers the whole spectrum of current submission systems capabilities, but it also provides a decentralised solution using open-source tools such as Java Spring that enhance the anonymity of the reviewing process.

show abstract

Boon, bias or bane? The potential influence of reviewer recommendations on editorial decision-making

Cited by 3 publications

References 19 publications

The limitations to our understanding of peer review

The limitations to our understanding of peer review

‘Conditional Acceptance’ (additional experiments required): A scoping review of recent evidence on key aspects of Open Peer Review

A Blockchain Cloud Computing Middleware for Academic Manuscript Submission

Contact Info

Product

Resources

About