Single-Blind and Double-Blind Peer Review: Effects on National Representation

Kalavar, Meghana; Watane, Arjun; Wu, David Bin-Chia; Sridhar, Jayanth; Mruthyunjaya, Prithvi; Parikh, Ravi

doi:10.1080/08820538.2021.1896757

Cited by 3 publications

(2 citation statements)

References 18 publications

(32 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When publishers require double-blind reviewing as standard, it helps set everyone on a more level playing field, benefitting both women and scholars from the Global South. While some studies indicate that double-blind does not necessarily benefit female authors (Cox & Montgomerie, 2019) or countries outside the US and UK (Kalavar et al, 2021), this remains an open debate. Studies do show that single-blind reviewing benefits well-known authors from elite institutions (Tomkins et al, 2017).…”

Section: What Can Scholars Do?mentioning

confidence: 99%

Equity in climate scholarship: a manifesto for action

Schipper¹,

Ensor²,

Mukherji³

et al. 2021

Climate and Development

View full text Add to dashboard Cite

Section: What Can Scholars Do?mentioning

confidence: 99%

Equity in climate scholarship: a manifesto for action

Schipper¹,

Ensor²,

Mukherji³

et al. 2021

Climate and Development

View full text Add to dashboard Cite

“…However, the coexistence of SB and DB reviewing (sometimes in the same venue) opens the door to comparative studies of the effects of prestige bias. For example, the association between reviewing policy and various biases has been assessed in disciplines as disparate as economics [4], behavioral ecology [5,6], ophthalmology [7], medicine [8,9], psychology [10], and computer science [3,11]. Some of these studies have found conclusive evidence for bias while others have found none, sometimes even in the same subfield.…”

Section: Introductionmentioning

confidence: 99%

Metrics and methods in the evaluation of prestige bias in peer review: A case study in computer systems conferences

Frachtenberg

McConville

2022

PLoS ONE

View full text Add to dashboard Cite

The integrity of peer review is essential for modern science. Numerous studies have therefore focused on identifying, quantifying, and mitigating biases in peer review. One of these better-known biases is prestige bias, where the recognition of a famous author or affiliation leads reviewers to subconsciously treat their submissions preferentially. A common mitigation approach for prestige bias is double-blind reviewing, where the identify of authors is hidden from reviewers. However, studies on the effectivness of this mitigation are mixed and are rarely directly comparable to each other, leading to difficulty in generalization of their results. In this paper, we explore the design space for such studies in an attempt to reach common ground. Using an observational approach with a large dataset of peer-reviewed papers in computer systems, we systematically evaluate the effects of different prestige metrics, aggregation methods, control variables, and outlier treatments. We show that depending on these choices, the data can lead to contradictory conclusions with high statistical significance. For example, authors with higher h-index often preferred to publish in competitive conferences which are also typically double-blind, whereas authors with higher paper counts often preferred the single-blind conferences. The main practical implication of our analyses is that a narrow evaluation may lead to unreliable results. A thorough evaluation of prestige bias requires a careful inventory of assumptions, metrics, and methodology, often requiring a more detailed sensitivity analysis than is normally undertaken. Importantly, two of the most commonly used metrics for prestige evaluation, past publication count and h-index, are not independent from the choice of publishing venue, which must be accounted for when comparing authors prestige across conferences.

show abstract

Impact of peer review type and editor gender on women authorships in mathematics

Lozano,

MacArthur,

Barr

2024

Quantitative Science Studies

View full text Add to dashboard Cite

Aiming to explain and help reduce measured gender gaps in mathematics publications—a discipline where single anonymized peer review practices and men editors are still the norm—we study changes in authorships straddling a 2015 switch in peer review type and editor gender in the American Mathematical Monthly, one of the oldest mathematics journals in the United States. Our results show a significant increase in women authorships after the journal’s switch to a woman editor and double anonymized reviews, an increase deemed exceptional relative to growth trends in comparable journals operating under field standards (men editors along with single anonymized reviews). We leverage literature, data-based observations and our own findings to argue for the likely effects of editor gender and peer review type separately and together as it concerns both women and newcomer authorships. Our study, the first to our knowledge on editorial and peer review changes in mathematics, contributes to a small body of literature on the impact of equity practices in peer review in science and mathematics. It also serves to introduce work, methods, and open problems on measuring and reducing the global gender gap in science and mathematics.

show abstract

Single-Blind and Double-Blind Peer Review: Effects on National Representation

Cited by 3 publications

References 18 publications

Equity in climate scholarship: a manifesto for action

Equity in climate scholarship: a manifesto for action

Metrics and methods in the evaluation of prestige bias in peer review: A case study in computer systems conferences

Impact of peer review type and editor gender on women authorships in mathematics

Contact Info

Product

Resources

About