A prospective study on an innovative online forum for peer reviewing of surgical science

Almquist, Martin; Allmen, Regula S. von; Carradice, Daniel; Oosterling, Steven J.; McFarlane, Kirsty; Wijnhoven, Bas P. L.

doi:10.1371/journal.pone.0179031

Cited by 5 publications

(3 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Indeed, we did not find any significant negative effects on referees’ willingness to review, their recommendations, or turn-around time. This contradicts recent research on individual cases, in which various forms of open peer review had a negative effect on these same factors 16,20 . Here, only younger and non-academic referees were slightly sensitive to the pilot.…”

Section: Discussioncontrasting

confidence: 78%

The effect of publishing peer review reports on referee behavior in five scholarly journals

et al. 2019

View full text Add to dashboard Cite

To increase transparency in science, some scholarly journals are publishing peer review reports. But it is unclear how this practice affects the peer review process. Here, we examine the effect of publishing peer review reports on referee behavior in five scholarly journals involved in a pilot study at Elsevier. By considering 9,220 submissions and 18,525 reviews from 2010 to 2017, we measured changes both before and during the pilot and found that publishing reports did not significantly compromise referees’ willingness to review, recommendations, or turn-around times. Younger and non-academic scholars were more willing to accept to review and provided more positive and objective recommendations. Male referees tended to write more constructive reports during the pilot. Only 8.1% of referees agreed to reveal their identity in the published report. These findings suggest that open peer review does not compromise the process, at least when referees are able to protect their anonymity.

show abstract

Section: Discussioncontrasting

confidence: 78%

The effect of publishing peer review reports on referee behavior in five scholarly journals

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Moreover, referees and authors who participated in cooperative interactions had a reviewing accuracy rate that was 11% higher. On the other hand, the possibility of publishing the reviews online has also been associated with a high decline rate among potential peer reviewers, and an increase in the amount of time taken to write a review, but with a variable effect on review quality ( Almquist et al , 2017 ; van Rooyen et al , 2010 ). This suggests that the barriers to publishing review reports are inherently social, rather than technical.…”

Section: The Traits and Trends Affecting Modern Peer Reviewmentioning

confidence: 99%

A multi-disciplinary perspective on emergent and future innovations in peer review

et al. 2017

View full text Add to dashboard Cite

Peer review of research articles is a core part of our scholarly communication system. In spite of its importance, the status and purpose of peer review is often contested. What is its role in our modern digital research and communications infrastructure? Does it perform to the high standards with which it is generally regarded? Studies of peer review have shown that it is prone to bias and abuse in numerous dimensions, frequently unreliable, and can fail to detect even fraudulent research. With the advent of web technologies, we are now witnessing a phase of innovation and experimentation in our approaches to peer review. These developments prompted us to examine emerging models of peer review from a range of disciplines and venues, and to ask how they might address some of the issues with our current systems of peer review. We examine the functionality of a range of social Web platforms, and compare these with the traits underlying a viable peer review system: quality control, quantified performance metrics as engagement incentives, and certification and reputation. Ideally, any new systems will demonstrate that they out-perform and reduce the biases of existing models as much as possible. We conclude that there is considerable scope for new peer review initiatives to be developed, each with their own potential issues and advantages. We also propose a novel hybrid platform model that could, at least partially, resolve many of the socio-technical issues associated with peer review, and potentially disrupt the entire scholarly communication system. Success for any such development relies on reaching a critical threshold of research community engagement with both the process and the platform, and therefore cannot be achieved without a significant change of incentives in research environments.

show abstract

“…. Although this approach has drawbacks (Almquist et al, 2017), the idea that the review products are subject to evaluation and improvement remains appealing. In the interest of improving the performance of reviewers, the scholarly community might consider developing strategies for evaluating the effectiveness of peer reviewers.…”

mentioning

confidence: 99%

From the Editor—On Peer Review: Improving the Credibility of Social Work Scholarship

Yaffe

2017

Journal of Social Work Education

View full text Add to dashboard Cite

Scholarly peer review is the process of having research and other scholarship submitted for the scrutiny of experts in the field to determine their opinion. JSWE and most other academic journals use blinded peer review to advise the editor on the relevance, coherence, and credibility of submitted manuscripts. We keep a list of more than 200 reviewers who represent a variety of social work programs, interest areas, and methodological foci. To be a good reviewer, one must have the willingness to perform a review and the time to perform that task carefully and completely. Further, reviewers must have broad and deep knowledge of at least one problem area or at least one methodological approach (preferably both), enough selfawareness to remain unbiased in their criticisms of blinded manuscripts, sufficient writing skills to explain in detail how manuscripts need to be revised, and enough patient beneficence to word their criticisms as kindly as possible. I am extremely grateful to the many reviewers who consistently respond to our requests to review manuscripts, complete the reviews on time, and write thorough reviews that focus on a manuscript's potential contribution to the field, its grounding in previous social work scholarship, the rigor and reproducibility of methods, and the logic of exposition. On the other hand, some contributors who have submitted manuscripts to JSWE wait a longer time than they, or we, would like before learning whether the paper has been accepted. When I assumed the role of editor-in-chief, one of my personal goals for the journal was to reduce the time to make decisions because social work scholars, and in particular pretenure faculty members, need to be able to see their publications accepted for publication and in print. Although I have made some progress in this regard, we still experience some dismaying delays in the review progress, and much of this delay is because of the difficulty in finding reviewers for a given manuscript. Sometimes we need to ask more than five people on our list of reviewers before we can find someone to accept a manuscript based on the title and the abstract. Occasionally we need to ask even more potential reviewers if they will accept a manuscript. Why is it sometimes so difficult to find a reviewer? Most of us would agree that faculty are busy people, and being a journal reviewer is not a priority. It might be that a manuscript does not seem interesting or might not be firmly in one's substantive or methodological areas of expertise. If these reasons explain the difficulty in locating reviewers, then JSWE staff need to find ways to increase our pool of reviewers. Even if all social work faculty members served a 3-year term as a reviewer, the question of whether service as a reviewer is more work than it is worth would remain. Although many of us derive intrinsic rewards for reviewing manuscripts, such as keeping up with current research or seeing other reviewers' comments about a manuscript, serving as a reviewer is usually seen by our fellow faculty member...

show abstract

A prospective study on an innovative online forum for peer reviewing of surgical science

Cited by 5 publications

References 24 publications

The effect of publishing peer review reports on referee behavior in five scholarly journals

The effect of publishing peer review reports on referee behavior in five scholarly journals

A multi-disciplinary perspective on emergent and future innovations in peer review

From the Editor—On Peer Review: Improving the Credibility of Social Work Scholarship

Contact Info

Product

Resources

About