Biomedical Journals Ponder the Failures and Remedies of Peer Review

Building bridges: knowledge production, publication and use. Commentary on Tonelli (2006), Integrating evidence into clinical practice: an alternative to evidence‐based approaches. Journal of Evaluation in Clinical Practice 12, 248–256

“…Integrity in this process was a subject of concern at The Fourth International Congress on Peer Review in Biomedical Publication, Barcelona, in 2001. At the meeting, participants looked for ‘ways to ensure that the process of peer review provides essential quality control in the publication of new research findings’ (Stephenson 2001, p. 2931). Drummond Rennie, MD, deputy editor of JAMA , noted that the study of peer review is a ‘new science’.…”

“…Drummond Rennie, MD, deputy editor of JAMA , noted that the study of peer review is a ‘new science’. Stephenson suggests ‘peer review is experiencing predictable growing pains as those who practise it attempt to uncover flaws and devise ways to improve the process’ (Stephenson 2001, p. 2931). This elementary status of the new science indicates problems inherent in relying on peer review to provide a sure ground, at least for the foreseeable future.…”

“…As is made clear, the published paper ‘rarely includes this range of opinion’ and most marked is the absence of known weaknesses. Stephenson (2001), who reported on the Barcelona meeting, says that given the limitations in published findings of research: ‘it is not surprising that authors sometimes fail to discuss results from similar trials conducted by other research groups and to explain how their own findings fit into the context of previous work’ (p. 2931).…”

“…Chalmers and colleagues base their opinion on two studies they conducted in 1997 and 2000. One analysed 26 reports of randomized trials in five ‘top medical journals’ published in May 1997 and found ‘only two reports discussed how the new findings compared with those of similar studies’ (Stephenson 2001, p. 2931). When they repeated the study, analysing 33 trials published in May 2001, they found similar omissions.…”

“…And, in light of other recent works cited in this commentary, extreme caution should be exercised before uncritically integrating empirical evidence derived from clinical trials, even when the practical know‐how exists. Stephenson (2001, p. 2931) argues that ‘in the modern era of evidence‐based medicine, medical journals and those who oversee them provide a critical bridge between research and practice’. Perhaps so, but with research quality, conflict of interest and peer review in question, such comments are thrown into doubt.…”