Practical Computational Reproducibility in the Life Sciences

Abstract

Many areas of research suffer from poor reproducibility, particularly in computationally intensive domains where results rely on a series of complex methodological decisions that are not well captured by traditional publication approaches. Various guidelines have emerged for achieving reproducibility, but implementation of these practices remains difficult due to the challenge of assembling software tools plus associated libraries, connecting tools together into pipelines, and specifying parameters. Here, we discuss a suite of cutting-edge technologies that make computational reproducibility not just possible, but practical in both time and effort. This suite combines three well-tested components—a system for building highly portable packages of bioinformatics software, containerization and virtualization technologies for isolating reusable execution environments for these packages, and workflow systems that automatically orchestrate the composition of these packages for entire pipelines—to achieve an unprecedented level of computational reproducibility. We also provide a practical implementation and five recommendations to help set a typical researcher on the path to performing data analyses reproducibly.

Successes and struggles with computational reproducibility: Lessons from the Fragile Families Challenge

“…Based on our experience with the special issue-as well as the suggestions of others Peng, 2009;Barnes, 2010;Sandve et al, 2013;Fehr et al, 2016;Bowers and Voors, 2016;Stodden et al, 2016;Eubank, 2016;Wilson et al, 2017;Marwick et al, 2018;Alvarez et al, 2018;Grning et al, 2018;Benureau and Rougier, 2018;Tatman et al, 2018;Konkol et al, 2019)-we have some recommendations for authors who wish to improve the computational reproducibility of their results now and in the future. We believe that these recommendations are applicable to all authors, even if they are not using software containers and cloud computing.…”

“…The benefits of computational reproducibility-and increased access to data and code, more generally-have already been articulated many times by researchers in many different fields: archaeology (Marwick, 2017), bioinformatics (Mangul et al, 2018), cell biology (Grning et al, 2018), computational fluid mechanics (Mesnard and Barba, 2017), computer systems research (Collberg and Proebsting, 2016), economics (Anderson et al, 2008;Koenker and Zeileis, 2009;Orozco et al, 2018), epidemiology (Peng et al, 2006;Coughlin, 2017;Shepherd et al, 2017), geosciences (Claerbout and Karrenbach, 1992;Gil et al, 2016;Konkol et al, 2019), high-energy physics (Chen et al, 2018), hydrology (Hutton et al, 2016), mathematical and computational biology (Schnell, 2018), machine learning (Tatman et al, 2018;Hutson, 2018), neuroscience (Crook et al, 2013;Manninen et al, 2017;Eglen et al, 2017;Mikowski et al, 2018), political science (King, 1995;Lupia and Elman, 2014;Alvarez et al, 2018), psychology (Clyburne-Sherin of the child, material hardship of the household, whether the household was evicted from their home, whether the primary caregiver participated in job training, and whether the primary caregiver lost his or her job. The choice of these outcomes was driven by our scientific goals and ethical considerations; each outcome is described in more detail elsewhere Lundberg et al, 2018).…”

Publishing computational research – A review of infrastructures for reproducible and transparent scholarly communication

“…Regarding submission formats, there is a trend towards literate programming approaches. Most applications either support Jupyter Notebook or R Markdown which both have proven to support reproducibility (Grüning et al 2018). However, some journals and publishers have particular requirements, e.g., they rely on LaTeX.…”

Scalable Data Analysis in Proteomics and Metabolomics Using BioContainers and Workflows Engines

“…Reproducibility is challenging in life sciences, especially in computationally intensive domains (e.g., proteomics and metabolomics) where results rely on a series of complex analytical and bioinformatics steps that are not well captured by traditional publication approaches. While there are now several guidelines and platforms to enable reproducibility in computational biology, the approach we describe here is flexible, robust, and scalable enough to guarantee the features for reproducibility research: i) managing software dependencies, ii) separation between the data flow design and the execution environments, and iii) virtualizing entire analyses for complete portability and preservation against time …”