A scoping review on biomedical journal peer review guides for reviewers

Song, Eunhye; Ang, Lin; Park, Ji-Yeun; Jun, Eun-Young; Kim, Kyeong Han; Jun, Jihee; Park, Sunju; Lee, Myeong Soo

doi:10.1371/journal.pone.0251440

Cited by 26 publications

(24 citation statements)

References 77 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…That said, the diversity of comments included in this section, and the fact that 51% of reviewers did not use the confidential comments area, suggests that journals might benefit from giving clearer guidance or training to their reviewer pool. Many journals provide guidance and checklists to reviewers, the contents of which were catalogued in a recent scoping review of biomedical journal peer review guides [7]. Unfortunately, this review did not report the number of guides that included content about how to use the confidential comments section, but may be worth exploring to understand the norms around use of this section and to help standardize language across peer review guides.…”

Section: Discussionmentioning

confidence: 99%

“…However, the soundness of this approach has come into question as such masking may inadvertently encourage reviewers to be less conscientious about providing constructive and respectful feedback to authors [6]. To mitigate this concern, many journals provide guidelines for reviewers that set expectations for quality feedback [7,8]. In addition, to enhance transparency and accountability, some journals have an open peer-review process that either requires or gives reviewers the option to share their identity and/or make their review publicly available [4,9].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Transparency in peer review: Exploring the content and tone of reviewers’ confidential comments to editors

et al. 2021

View full text Add to dashboard Cite

Purpose Recent calls to improve transparency in peer review have prompted examination of many aspects of the peer-review process. Peer-review systems often allow confidential comments to editors that could reduce transparency to authors, yet this option has escaped scrutiny. Our study explores 1) how reviewers use the confidential comments section and 2) alignment between comments to the editor and comments to authors with respect to content and tone. Methods Our dataset included 358 reviews of 168 manuscripts submitted between January 1, 2019 and August 24, 2020 to a health professions education journal with a single blind review process. We first identified reviews containing comments to the editor. Then, for the reviews with comments, we used procedures consistent with conventional and directed qualitative content analysis to develop a coding scheme and code comments for content, tone, and section of the manuscript. For reviews in which the reviewer recommended “reject,” we coded for alignment between reviewers’ comments to the editor and to authors. We report descriptive statistics. Results 49% of reviews contained comments to the editor (n = 176). Most of these comments summarized the reviewers’ impression of the article (85%), which included explicit reference to their recommended decision (44%) and suitability for the journal (10%). The majority of comments addressed argument quality (56%) or research design/methods/data (51%). The tone of comments tended to be critical (40%) or constructive (34%). For the 86 reviews recommending “reject,” the majority of comments to the editor contained content that also appeared in comments to the authors (80%); additional content tended to be irrelevant to the manuscript. Tone frequently aligned (91%). Conclusion Findings indicate variability in how reviewers use the confidential comments to editor section in online peer-review systems, though generally the way they use them suggests integrity and transparency to authors.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Transparency in peer review: Exploring the content and tone of reviewers’ confidential comments to editors

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Many journals provide guidance and checklists to reviewers, the contents of which were catalogued in a recent scoping review of biomedical journal peer review guides. [7] Unfortunately, this review did not report the number of guides that included content about how to use the confidential comments section, but may be worth exploring to understand the norms around use of this section and to help standardize language across peer review guides.…”

Section: Discussionmentioning

confidence: 99%

“…[6] To mitigate this concern, many journals provide guidelines for reviewers that set expectations for quality feedback. [7,8] In addition, to enhance transparency and accountability, some journals have an open peer-review process that either requires or gives reviewers the option to share their identity and/or make their review publicly available [4,9]. While such efforts to increase transparency are thought to improve review quality by increasing reviewers' sense of accountability, a series of studies conducted by the BMJ found no difference in the quality of masked versus unmasked peer reviews.…”

Section: Introductionmentioning

confidence: 99%

Transparency in peer review: Exploring the content and tone of reviewers’ confidential comments to editors

O’Brien

Artino

Costello

et al. 2021

Preprint

View full text Add to dashboard Cite

Purpose: Recent calls to improve transparency in peer review have prompted examination of many aspects of the peer review process. Peer review systems often allow confidential comments to editors that could reduce transparency to authors, yet this option has escaped scrutiny. Our study explores 1) how reviewers use the confidential comments section and 2) alignment between comments to the editor and comments to authors with respect to content and tone. Methods: Our dataset included 358 reviews of 168 manuscripts submitted between January 1, 2019 and August 24, 2020 to a health professions education journal with a single blind review process. We first identified reviews containing comments to the editor. Then, for the reviews with comments, we used procedures consistent with conventional and directed qualitative content analysis to develop a coding scheme and code comments for content, tone, and section of the manuscript. For reviews in which the reviewer recommended "reject," we coded for alignment between reviewers' comments to the editor and to authors. We report descriptive statistics. Results: 49% of reviews contained comments to the editor (n=176). Most of these comments summarized the reviewers' impression of the article (85%), which included explicit reference to their recommended decision (44%) and suitability for the journal (10%). The majority of comments addressed argument quality (56%) or research design/methods/data (51%). The tone of comments tended to be critical (40%) or constructive (34%). For the 48 reviews recommending "reject," the majority of comments to the editor contained content that also appeared in comments to the authors (65%); additional content tended to be irrelevant to the manuscript. Tone frequently aligned (85%). Conclusion: Findings indicate variability in how reviewers use the confidential comments to editor section in online peer review systems, though generally the way they use them suggests integrity and transparency to authors.

show abstract

“…A quality assessment limitation will also be implemented during the search stage. Although quality or critical assessment of studies is not a scoping review requirement [ 56 , 57 ], some published scoping review protocols [ 68 , 69 ] and reviews [ 70 – 73 ] include such an assessment. Further, as one of the purposes of the proposed review is to identify gaps in the literature, quality assessment has been recommended in such situations [ 74 ].…”

Section: Methodsmentioning

confidence: 99%

Online suicidal thoughts and/or behaviours talk: A scoping review protocol

Lamont‐Mills

Bayliss²,

Christensen³

2022

PLoS ONE

View full text Add to dashboard Cite

The anonymity that the internet and social media affords users means that suicidal thoughts and/or behaviours can be talked about with a sense of freedom and disinhibition that is often not possible in face-to-face contexts. Better understanding online suicidal thoughts and/or behaviour talk is critical as more people turn to online spaces for support. Without this the potentiality of such spaces as sites for suicide prevention and intervention is likely to remain unrealised. Currently there are no scoping or systematic review syntheses focusing on internet and/or on social media suicidal thoughts and/or behaviour talk. This lack of synthesis is problematic as it makes it more difficult for online suicide prevention and intervention practices, policies, and our understanding of suicide to advance in a coherent and evidence-based manner. A scoping review protocol following Arksey and O’Malley’s six-step modified framework has been developed to address this synthesis gap. It aims to systematically map the empirical literature that has investigated online suicidal thoughts and/or behaviours talk. It is anticipated that review outcomes could inform the training of health practitioners and peer/professional online moderators in how to best talk with people experiencing suicidal thoughts and/or behaviours. Outcomes could also form an evidence-base for developing policies and practices that focus on online places as safe spaces to talk about suicidal thoughts and/or behaviours. Developers of safe language guidelines could also use the outcomes to audit how well current guidelines reflect empirical evidence. Outcomes could enable researchers to design future online suicidal thoughts and/behaviours talk studies that extend our understandings of suicide leading to potential refinements of contemporary suicide theories/models.

show abstract

A scoping review on biomedical journal peer review guides for reviewers

Cited by 26 publications

References 77 publications

Transparency in peer review: Exploring the content and tone of reviewers’ confidential comments to editors

Transparency in peer review: Exploring the content and tone of reviewers’ confidential comments to editors

Transparency in peer review: Exploring the content and tone of reviewers’ confidential comments to editors

Online suicidal thoughts and/or behaviours talk: A scoping review protocol

Contact Info

Product

Resources

About