Reproducibility of clinical research in critical care: a scoping review

Niven, Daniel J.; McCormick, Thomas Jared; Straus, Sharon E.; Hemmelgarn, Brenda R.; Jeffs, Lianne; Barnes, Thomas A; Stelfox, Henry T.

doi:10.1186/s12916-018-1018-6

Cited by 63 publications

(49 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another study compared the results from a limited number of initial clinical studies and respective follow-up studies. It concluded that less than 50% of the investigated studies reported reproducible effects [49]. However, it is not clear how representative the data are.…”

Section: Resultsmentioning

confidence: 99%

Understanding of researcher behavior is required to improve data reliability

2019

View full text Add to dashboard Cite

Background A lack of data reproducibility (“reproducibility crisis”) has been extensively debated across many academic disciplines. Results Although a reproducibility crisis is widely perceived, conclusive data on the scale of the problem and the underlying reasons are largely lacking. The debate is primarily focused on methodological issues. However, examples such as the use of misidentified cell lines illustrate that the availability of reliable methods does not guarantee good practice. Moreover, research is often characterized by a lack of established methods. Despite the crucial importance of researcher conduct, research and conclusive data on the determinants of researcher behavior are widely missing. Conclusion Meta-research that establishes an understanding of the factors that determine researcher behavior is urgently needed. This knowledge can then be used to implement and iteratively improve measures that incentivize researchers to apply the highest standards, resulting in high-quality data.

show abstract

Section: Resultsmentioning

confidence: 99%

Understanding of researcher behavior is required to improve data reliability

2019

View full text Add to dashboard Cite

show abstract

“…Precision medicine improves patient diagnoses and treatments by integrating clinical and biomarker data, which requires effective data management for reproducibility of clinical research findings (Vargas and Harris, 2016; Leopold and Loscalzo, 2018; Niven et al , 2018). The studies often span institutions, departments, and research teams, where patient data is collected and processed in a specialized way.…”

Section: Introductionmentioning

confidence: 99%

cohorts: A Python package for clinical ‘omics data management

Fine

Tatonetti

2019

Preprint

View full text Add to dashboard Cite

6Summary: Precision medicine uses patient clinical and molecular characteristics to personalize diagnosis 7 and treatment. This emerging discipline integrates multi-modal data into large-scale studies of human 8 disease to make accurate individual-level predictions. The success of these studies will depend on the 9 generalizability of the results, the ability of other researchers and clinicians to replicate studies, and the 10 understandability of the methods used. Tools for data management and standardization are needed to 11 promote flexible, transparent, and reproducible analyses. Here we present cohorts, a python package 12 facilitating clinical and biomarker data management to enhance standardization and reproducibility of 13 clinical findings. 14 Availability: The python package cohorts is available at http://www.github.com/ngiangre/cohorts. 15 CONTACT: NPT2105@CUMC.COLUMBIA.EDU 16 17 19 requires effective data management for reproducibility of clinical research findings (Vargas and Harris, 2016; Leopold20 and Loscalzo, 2018; Niven et al., 2018). The studies often span institutions, departments, and research teams, where 21 patient data is collected and processed in a specialized way. Clinical studies including patient data from multiple 22 sources are integrated and managed by the primary research team, and so correct attribution of the clinical and 23 biomarker characteristics is critical. Feasible and accessible software can facilitate data storage and management, 24 which promotes flexible, transparent, and reproducible analyses. Moreover, software that facilitates integration of 25 multiple patient cohorts can promote the use of advanced statistical and machine learning analyses. 26Precision medicine calls for rigor and reproducibility in data generation and data analysis (Mueller et al., 2018; 27 Orton and Doucette, 2013). For large consortium based studies, computational platforms allow for sharing data and 28 merging of multisite datasets (Lam et al., 2016). While large-scale studies may have an extensive computing platform 29 with custom software (Wolstencroft et al., 2015; Price et al., 2019), most clinical studies are smaller and would benefit 30 from standardized procedures and interoperable data management and analysis tools. There are many software 31 packages and platforms that perform precision medicine data analysis, such as pyGeno (Daouda et al., 2016) and in 32 particular packages from the package manager Bioconductor in the R programming language (Huber et al., 2015). 33However, many of these applications can handle only one experimental type or are particular to a specific end-to-end 34 data analysis. There is a need for flexible and open source software, where patient clinical and biomarker data can 35 originate from different cohorts, platforms, and data types. 36Here, we present the python package cohorts, which provides storage and management of patient clinical and 37 biomarker data. Moreover, cohorts facilitates integration of multiple patient cohort data through common attrib...

show abstract

“…A recent scoping review of the reproducibility of research in critical care found that less than half of critical care practices had reproducible effects, and worse, two practices that were initially efficacious were found to be harmful in repeated studies (Niven et al., ), a major reason why it is so important to conduct replication research. Further, the challenges of reproducibility in research are not limited to the United States.…”

mentioning

confidence: 99%