Reproducibility and transparency characteristics of oncology research evidence

Abstract

Introduction: As much as 50%-90% of research is estimated to be irreproducible, costing upwards of $28 billion in the United States alone. Reproducible research practices are essential to improving the reproducibility and transparency of biomedical research, such as including pre-registering studies, publishing a protocol, making research data and metadata publicly available, and publishing in open access journals. Here we report an investigation of key reproducible or transparent research practices in the published oncology literature. Methods: We performed a cross-sectional analysis of a random sample of 300 oncology studies published from 2014-2018. We extracted key reproducibility and transparency characteristics in a duplicative fashion by blinded investigators using a pilot tested Google Form. Results: Of the 300 studies randomly sampled, 296 studies were analyzed for study reproducibility characteristics. Of these 296 studies, 194 were contained empirical data that could be analyzed for reproducible and transparent research practices. Raw data was available for 9 studies (4.6%). Approximately 5 studies (2.6%) provided a protocol. Despite our sample including 15 clinical trials and 7 systematic reviews/meta-analyses, only 7 included a pre-registration statement. Less than 25% (65/194) of studies provided an author conflict of interest statement. Discussion: We found that key reproducibility and transparency characteristics were absent from a random sample of published oncology studies. We recommend required pre-registration for all eligible trials and systematic reviews, published protocols for all manuscripts, and deposition of raw data and metadata in public repositories.

Where Do Early Career Researchers Stand on Open Science Practices? A Survey Within the Max Planck Society

“…This is reflected in our data, where preregistrations and registered reports were relatively more known, more positively evaluated, and more frequently used among ECRs from the HUM section than those from the BM and CPT sections. However, this could be likely to change in future years, with researchers from fields other than the social sciences calling for the implementation of preregistrations and registered reports to improve the transparency and reproducibility of their research fields (e.g., Parker et al, 2019 ; Walters et al, 2019 ). We further observed that ECRs across sections showed similarly low levels of participation in replication studies.…”

Authorial Conflicts of Interest and Sponsorship in Systematic Reviews and Meta-Analyses on Psoriasis

“…The reproducibility of systematic reviews, however, is very difficult if not impossible to undertake. The senior author of this study widely evaluates research reproducibility [41][42][43][44][45][46][47][48][49] and has evaluated the reproducibility of meta-analyses. 50 While meta-analytic effect estimates may be more reproducible in some cases, earlier steps of the systematic review process are likely not.…”