Single-blind vs Double-blind Peer Review in the Setting of Author Prestige

Abstract

Most medical journals practice single-blind review 1 (authors' identities known to reviewers), but double-blind review (authors' identities masked to reviewers) may improve the quality of reviews. 2 Bias with single-blind review might be greatest in the setting of author or institutional prestige. 2 Methods | This study was approved by the University of Washington institutional review board and conducted at Clinical Orthopaedics and Related Research (CORR), an orthopedic journal (2015 impact factor, 3.127; acceptance rate, 20%) that allows authors to select single-blind or double-blind peer review. Potential reviewers (based on expertise) were identified from the journal's database, informed that a study on peer review would occur in the coming year, and allowed to opt out. To avoid influencing behavior, the details and timing of the study were not described and the trial was registered after completion. The protocol appears in the Supplement. Between June 2014 and August 2015, reviewers were randomized via random number table (1:1 ratio in blocks of 8) to receive single-blind or double-blind versions of an otherwise identical fabricated manuscript, which was putatively writ

Editorial: CORR ® Will Change to Double-blind Peer Review—What Took Us So Long to Get There?

“…In addition, I am grateful to Daniel J. Berry MD (of the Mayo Clinic) and James H. Herndon MD, MBA (of Harvard University) for lending their ''identities'' to the research project [11]. Finally, I appreciate the guidance from CORR's panel of Senior Editors on the topics of whether and how to convert the findings of the experiment into editorial policy here; these Editors are …”

“…In the randomized trial conducted at CORR, and published recently in JAMA [11], two versions of a fabricated manuscript were sent out to several hundred peer reviewers. The two versions were identical except that in one version the names of well-known authors from prestigious institutions were visible to reviewers, while in the other version reviewers were blinded to authors' identities and universities.…”

“…The controversies on this topic, the experiment's somewhat-troubling findings [11], and the fact that even a thoughtfully arrived-at policy is unlikely to eliminate fully even this one kind of bias (from among the numerous others that certainly remain) highlight how very complicated peer review is, and how difficult it is to do well.…”

The Case For and Against Double-blind Reviews

“…There is evidence, for example, that authors familiar to reviewers, either through a personal connection or prominence in the field, are more likely to have their papers or grants accepted than unfamiliar authors (Sandstrom & Hallsten, 2008; Okike et al, 2016). In Sweden, success rates for medical grants were ~15% higher when the grant committee members were personally affiliated with the applicant (Sandstrom & Hallsten, 2008).…”

“…Second, real or fake manuscripts can be assigned randomly to multiple readers to assess the effects of different author-gender combinations on perceived quality (e.g., Borsuk et al, 2009; Knobloch-Westerwick, Glynn & Huge, 2013; Okike et al, 2016). This method is excellent with respect to experimental design as so many potentially confounding factors can be controlled but it requires a fairly large number of willing and knowledgeable readers.…”