Peer Review: A Flawed Process at the Heart of Science and Journals

Abstract

Richard Smith was editor of the BMJ and chief executive of the BMJ Publishing Group for 13 years. In his last year at the journal he retreated to a 15th century palazzo in Venice to write a book. The book will be published by RSM Press [www.rsmpress.co.uk], and this is the second in a series of extracts that will be published in the JRSM. DOES PEER REVIEW 'WORK' AND WHAT IS IT FOR? But does peer review 'work' at all? A systematic review of all the available evidence on peer review concluded that

Is there bias in editorial choice? Yes

“…is clearly Yes. It is confirmed by many recent and old reports (Manchikanti et al 2015;Phillips 2011;Smith 2006;Lush 2006;Eaton and Anthony 2000). If a corresponding author is not fame or 'star', another fame co-author in the authors' list would inevitably introduce some celebrity biases for the corresponding author himself.…”

Do editors and referees look for signs of scientific misconduct when reviewing manuscripts? A quantitative content analysis of studies that examined review criteria and reasons for accepting and rejecting manuscripts for publication

“…Despite the difficulties and expense of involved in reviewing underlying data, LEE & BERO [2006] suggest that having a policy of access to raw data would "hold authors more accountable for the accuracy of their data and potentially reduce scientific fraud or misconduct". Already today, some journals, including the British Medical Journal, make it a condition that the editors can 431 ask for the raw data on which the results of a manuscript submitted for publication are based [SMITH, 2006]. By the way, the Journal of the American Society for Information Science and Technology asks referees to look for suspicion of duplicate publication, fabrication of data or plagiarism.…”

Peer review in a rapidly evolving publishing landscape

“…The shortcomings of peer review have been highlighted (Wager and Jefferson, 2001) and criticisms levelled on a number of fronts (Smith, 2006(Smith, , 2010a: about its quality and fairness, that it is open to abuse and bias, that it is expensive, slow and lacks consistency, and that it is conservative. There are concerns about delays, both in the time taken for review and decision at individual journals, and in the time taken for a piece of work to be published because it sometimes has to go from journal to journal, being rejected a number of times before finally being accepted (a manuscript can only be sent to one journal at a time).…”