A checklist is associated with increased quality of reporting preclinical biomedical research: A systematic review

Abstract

Irreproducibility of preclinical biomedical research has gained recent attention. It is suggested that requiring authors to complete a checklist at the time of manuscript submission would improve the quality and transparency of scientific reporting, and ultimately enhance reproducibility. Whether a checklist enhances quality and transparency in reporting preclinical animal studies, however, has not been empirically studied. Here we searched two highly cited life science journals, one that requires a checklist at submission (Nature) and one that does not (Cell), to identify in vivo animal studies. After screening 943 articles, a total of 80 articles were identified in 2013 (pre-checklist) and 2015 (post-checklist), and included for the detailed evaluation of reporting methodological and analytical information. We compared the quality of reporting preclinical animal studies between the two journals, accounting for differences between journals and changes over time in reporting. We find that reporting of randomization, blinding, and sample-size estimation significantly improved when comparing Nature to Cell from 2013 to 2015, likely due to implementation of a checklist. Specifically, improvement in reporting of the three methodological information was at least three times greater when a mandatory checklist was implemented than when it was not. Reporting the sex of animals and the number of independent experiments performed also improved from 2013 to 2015, likely from factors not related to a checklist. Our study demonstrates that completing a checklist at manuscript submission is associated with improved reporting of key methodological information in preclinical animal studies.

A randomised controlled trial of an Intervention to Improve Compliance with the ARRIVE guidelines (IICARus)

“…Our results contrast with recent reports of improvement in quality following mandated checklist completion following a change in editorial policy at Nature journals (Han et al, 2017, Macleod, 2017. However, in both reports study quality was retrospectively assessed in publications published prior to and after the introduction of the Nature quality checklist, which was established in 2015 as part of an organisation wide approach with substantial editorial involvement.…”

“…Simply endorsing the guidelines does not appear to be sufficient to encourage compliance. Recent findings suggest that following the introduction of mandated completion of a distinct reporting checklist at ten Nature Journals at the stage of first revision significantly improved the quality in reporting versus that of comparator journals (Han et al, 2017, Macleod, 2017 PLOS ONE is an open access online only journal which at the time this study began published around 32,000 research articles per year. Of these, some 5,000 described in vivo research.…”

A Systematic Review of Automatic Question Generation for Educational Purposes

“…At a minimum, data extracted in this review (refer to questions under OBJ2 and OBJ3) should be reported in all publications on AQG. To ensure quality, journals can require authors to be complete a checklist prior to peer review, which has shown to improve the reporting quality (Han et al 2017). Alternatively, text-mining techniques can be used for assessing the reporting quality by targeting key information in AQG literature, as has been proposed in Flórez-Vargas et al (2016).…”

The ARRIVE guidelines 2019: updated guidelines for reporting animal research

“…A randomised controlled trial at PLOS ONE, for example, demonstrated that a request by journal staff to include a completed ARRIVE checklist in the manuscript submission process did not improve the disclosure of information in published papers [31]. In contrast, other studies using reporting checklists with more editorial follow up have shown a marked improvement in the nature and detail of the information included in publications [32][33][34]. Providing the level of journal or editorial input required to ensure compliance with all the items of the ARRIVE guidelines is unlikely to be sustainable for most journals because of the resources needed.…”