Improving the reliability of peer-reviewed publications: We are all in it together

Abstract

The current, and welcome, focus on standardization of techniques and transparency of reporting in the biomedical, peer-reviewed literature is commendable. However, that focus has been intermittent as well as lacklustre and so failed to tackle the alarming lack of reliability and reproducibly of biomedical research. Authors have access to numerous recommendations, ranging from simple standards dealing with technical issues to those regulating clinical trials, suggesting that improved reporting guidelines are not the solution. The elemental solution is for editors to require meticulous implementation of their journals’ instructions for authors and reviewers and stipulate that no paper is published without a transparent, complete and accurate materials and methods section.

qPCR primer design revisited

“…Given the significance of empirical validation, it is important that any publication include that essential information [8] , [9] , [10] , [11] . Several reports have been published recently that together scored thousands of peer-reviewed papers in a wide collection of journals ranging from low to high impact factors [12] , [13] , [14] , [15] , [16] , [17] . All concluded that the amount of critical information provided with papers reporting qPCR data is inadequate for the purpose of evaluating the validity of conclusions arising from those data, with many not reporting primer sequences, validation data or including wrong information.…”

Talking the talk, but not walking the walk: RT-qPCR as a paradigm for the lack of reproducibility in molecular research

“…Indeed, Nature has acknowledged that editorial policies have contributed to ‘failures in the reliability and reproducibility of published research’ and admitted that journals ‘compound’ them by failing to ‘exert sufficient scrutiny’ and not publishing ‘enough information for other researchers to assess results properly’ , a message reaffirmed by many other Nature titles . However, the suggested paths to action have remained largely untrodden, over the last 5 years peer‐reviewed papers of dubious quality have continued to be published in low and high impact factor journals alike and the scientific literature continues to be filled with thousands of papers that report results that are at best ambiguous and, at worst, simply wrong . Regrettably, to date, the promises from many publishing houses to improve the transparency of reporting remain unfulfilled.…”

“…Reproducibility incorporates both biological and technical variability, and as long ago as 1949, it was demonstrated that experimental test results can vary widely, even when performed by the same individual at the same time . Since then, there have been numerous publications that highlight the problems of lack of reproducibility (reviewed in ) and the role journals play in failing to enforce their own editorial policies . This, together with the fact that credibility and translation are only modestly correlated , explains why basic research findings are rarely adopted into clinical practice .…”

Unit of analysis issues in laboratory-based research

“…UoA issues , as they are termed in the statistical literature ( Altman and Bland, 1997 ), are not limited to biomedical laboratory studies, and are recognised as a major cause of concern more generally for reported analyses in bioscience and medicine ( Aarts et al, 2014 ; Altman and Bland, 1997 ; Bunce et al, 2014 ; Fleming et al, 2013 ; Lazic, 2010 ; Calhoun et al, 2008 ; Divine et al, 1992 ), and also feed into widely acknowledged issues around the lack of reproducibility and repeatability of much biomedical research ( Academy of Medical Sciences, 2017 ; Bustin and Nolan, 2016 ; Ioannidis et al, 2014 ; McNutt, 2014 ).…”