Facilitating the Reproducibility of Scientific Workflows with Execution Environment Specifications

Hu, Meng; Thain, Douglas

doi:10.1016/j.procs.2017.05.116

Cited by 14 publications

(7 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Full reproducibility and reusability of scientific results requires not only the data and code, but also the whole software environment in which the code was executed. There are several solutions to achieve reproducible execution environments 8 . We adopted incremental Reproducible Execution Environment Specifications (REES) to build a reproducible environment for the Atlas repository.…”

Section: Resultsmentioning

confidence: 99%

Implementation of FAIR principles in the IPCC: the WGI AR6 Atlas repository

et al. 2022

View full text Add to dashboard Cite

The Sixth Assessment Report (AR6) of the Intergovernmental Panel on Climate Change (IPCC) has adopted the FAIR Guiding Principles. We present the Atlas chapter of Working Group I (WGI) as a test case. We describe the application of the FAIR principles in the Atlas, the challenges faced during its implementation, and those that remain for the future. We introduce the open source repository resulting from this process, including coding (e.g., annotated Jupyter notebooks), data provenance, and some aggregated datasets used in some figures in the Atlas chapter and its interactive companion (the Interactive Atlas), open to scrutiny by the scientific community and the general public. We describe the informal pilot review conducted on this repository to gather recommendations that led to significant improvements. Finally, a working example illustrates the re-use of the repository resources to produce customized regional information, extending the Interactive Atlas products and running the code interactively in a web browser using Jupyter notebooks.

show abstract

Section: Resultsmentioning

confidence: 99%

Implementation of FAIR principles in the IPCC: the WGI AR6 Atlas repository

et al. 2022

View full text Add to dashboard Cite

show abstract

“…We emphasize the importance of generating workflows 3,26 , not only for tracing provenance of data and making them transparent to the community 27 , but also in order to explain in a detailed and compact way the scientific strategies used in the paper; these may then be used for training purposes for students or investigators interested in joining a specific project related to the paper, or for collaborations. In the future, the generation of the workflow can be automated by enabling Qresp to read the metadata produced by workflow management tools, e.g.…”

Section: Resultsmentioning

confidence: 99%

Qresp, a tool for curating, discovering and exploring reproducible scientific papers

et al. 2019

View full text Add to dashboard Cite

We propose a strategy and present a simple tool to facilitate scientific data reproducibility by making available, in a distributed manner, all data and procedures presented in scientific papers, together with metadata to render them searchable and discoverable. In particular, we describe a graphical user interface (GUI), Qresp, to curate papers (i.e. generate metadata) and to explore curated papers and automatically access the data presented in scientific publications.

show abstract

“…Annotating a workflow execution with the provenance of its components has been previously used for traceability, reproducibility, and explaining results. Related work for collecting and preserving the provenance of a workflow at the system level include developing custom file systems tracking provenance such as the Lineage File System (LinFS) [11], PASTA in PASS (Provenance-Aware Storage Systems) [8], and Parrot [12]; encapsulating workflows through ad hoc packages such as CDE [13], ReproZip [14], Umbrella [15], and Occam [16] [17]; and encapsulating workflows through existing container technologies such as Pachyderm [9], RE-ANA [10], and Science Capsule [41]. The use of custom file systems and custom ad hoc packages limits the portability and usability of their solutions across systems.…”

Section: Related Workmentioning

confidence: 99%

“…Second, scientists track provenance with custom systems that do not offer portability across heterogeneous platforms. For example, work in [8], [11], [12] builds on custom file systems, and work in [13], [14], [15], [16], [17] builds on custom software packaging. Containers offer a lightweight solution to track provenance across platforms by encapsulating applications and dependencies into an isolated environment [18], [19], [20], [21].…”

mentioning

confidence: 99%

Building Trust in Earth Science Findings through Data Traceability and Results Explainability

Olaya

Kennedy

Llamas

et al. 2023

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

To trust findings in computational science, scientists need workflows that trace the data provenance and support results explainability. As workflows become more complex, tracing data provenance and explaining results become harder to achieve. In this paper, we propose a computational environment that automatically creates a workflow execution's record trail and invisibly attaches it to the workflow's output, enabling data traceability and results explainability. Our solution transforms existing container technology, includes tools for automatically annotating provenance metadata, and allows effective movement of data and metadata across the workflow execution. We demonstrate the capabilities of our environment with the study of SOMOSPIE, an earth science workflow. Through a suite of machine learning modeling techniques, this workflow predicts soil moisture values from the 27 km resolution satellite data down to higher resolutions necessary for policy making and precision agriculture. By running the workflow in our environment, we can identify the causes of different accuracy measurements for predicted soil moisture values in different resolutions of the input data and link different results to different machine learning methods used during the soil moisture downscaling, all without requiring scientists to know aspects of workflow design and implementation.

show abstract

Facilitating the Reproducibility of Scientific Workflows with Execution Environment Specifications

Cited by 14 publications

References 13 publications

Implementation of FAIR principles in the IPCC: the WGI AR6 Atlas repository

Implementation of FAIR principles in the IPCC: the WGI AR6 Atlas repository

Qresp, a tool for curating, discovering and exploring reproducible scientific papers

Building Trust in Earth Science Findings through Data Traceability and Results Explainability

Contact Info

Product

Resources

About