Investigating journal peer review as scientific object of study: unabridged version – Part II

Abstract

The main goal of this paper is to construct journal peer review as a scientific object of study based on historical research into the shaping of its structural properties. This paper is a second in a two-part series. Journal peer review performed in the natural sciences has been an object of study since at least 1830. Researchers mostly implicitly frame it as a rational system with expectations of rational decision-making. This in spite of research debunking rationality where journal peer review can yield low inter-rater reliability, be purportedly biased and conservative, and cannot readily detect fraud or misconduct.

Investigating journal peer review as scientific object of study: unabridged version – Part I

“…In this first paper of a two-part series, I focus on the emergence, development, flourishing and dissolution of social forms. In the second part (Gaudet, 2014a), I focus on investigating structural properties for the social forms.…”

How peer-review constrains cognition: on the frontline in the knowledge sector

“…Individual activity occurs within domains such as farms and universities or what Hutchins (2014) calls ‘cognitive ecosystems’. On this view, one can look beyond the normative by allowing peer-review to include Gaudet’s (2014) ‘structural relations’ and Pontille and Torny’s (2015) ‘technology’. Using these models, Figure 2 , presents the two contrasting framings of peer-review.…”

“…In that it reduces peer-review to changes in ‘form’, appeal to institutionalization is anything but trivial. There is, of course, dissent; for Gould (2013) peer-review arose from ‘censorship and inquisition’ ( Gaudet, 2014 ) and, turning from a process view, while Hirschauer (2009) stresses reciprocal accountability, Gaudet (2014) underlines its bounding function. For Pontille and Torny (2015) , as a technology, peer-review ties evaluation to aggregated epistemic judgements (validation).…”

“…Since the 1980s, a consensus has arisen –as shared by, for example, Bedeian (2003) , Hirschauer (2009) , Bornmann (2011) , and Gaudet (2014) . Building on empirical extensions of Merton’s (1942) work, this is deeply influenced by biomedical concerns.…”

“…Lack of consensus can be traced to diverse positions, backgrounds, criteria of judgment and, occasionally, dissensus is valued ( Bailar, 1991 ). Finally, some think that a quest for reliability posits the wrong object of study ( Gaudet, 2014 ) and/or a distorted view of science. In any case, ‘good’ reviews, whatever those may be, are highly unlikely to be reliable.…”