From Manuscript Evaluation to Article Valuation: The Changing Technologies of Journal Peer Review

Abstract

International audienceBorn in the seventeenth century, journal peer review is an extremely diverse technology, constantly torn between two often incompatible goals: the validation of manuscripts conceived as a collective industrial-like reproducible process performed to assert scientific statements, and the dissemination of articles considered as a means to spur scientific discussion, raising controversies and civically challenging a state of knowledge. Such a situation is particularly conducive to clarifying the processes of valuation and evaluation in journal peer review. In this article, such processes are considered as specific tests in order to emphasize the uncertain properties of pre-tests manuscripts. On the one hand, evaluation tests are examined at the core of the validation of manuscripts, such as defining the coordination of judging instances (editor-in-chief, editorial committee, outside reviewers) or controlling the modalities of inter-knowledge between reviewers and authors. They are also studied regarding the dissemination of articles, notably through the contemporary conception of a continuing evaluation test termed " post publication peer review ". On the other hand, valuation tests are both part of the validation of manuscripts, such as the weighting of different judgments of the same manuscript and the tensions that these hierarchies cause, and of the dissemination of articles, such as attention metrics recording the uses of articles. The conclusion sketches out how the articulation of these different tests has recently empowered readers as a new key judging instance for dissemination and for validation, potentially transforming the definition of peers, and thus the whole process of journal peer review

Managing the Growth of Peer Review at the Royal Society Journals, 1865-1965

“…The socialization of the Fellows facilitated the construction and implementation of community-based standards for "publishable research" and academic merit (Pontille and Torny 2015), in the absence of explicit criteria for evaluation. For instance, in the 1880s, referees had been asked for "your opinion as regards its eligibility for publication in the Philosophical Transactions"; what counted as "eligibility" was assumed to be tacitly understood (e.g., Referee report, February 16, 1883, RR/9/171).…”

On the origin of nonequivalent states: How we can talk about preprints

“…Today, questions are raised as to whether processes that do not blind referees to author identity (a process described by specific state changes) can be expected to be unbiased and therefore valid (a question of standing). Pontille and Torny 5 in examining the complex history of these views quote Lowry 6 to showcase the view that “...a man’s[sic] name is important and...can be used as a basis for judging the reliability and relevance of what he says”. Separating the value-laden discussion of what judgements are necessary or reliable from the details of the process that support them can help to uncover and illuminate effective paths forward in deep-seated disagreements.…”

The role of the anonymous voice in post-publication peer review versus traditional peer review

“…PPPR appears to have established itself as one of the most robust tool to avoid COIs and serves as a mechanism to correct already published literature, as its primary function (Teixeira da Silva 2013Silva , 2015aSilva , 2015bSilva , 2015c. PPPR also has other functions: expanding the scientific discussion, including analysis, commentary and criticism (Pontille and Torny, 2015). PPPR allows for the public discussion of the merits or issues related to errors in the literature, including negative instances of duplicate data, plagiarism, or image manipulation, by a pool of peers or experts that extends beyond the limited number of individuals in the TPR model assigned to complete the task of quality control.…”