Building Trust in Earth Science Findings through Data Traceability and Results Explainability

Abstract

To trust findings in computational science, scientists need workflows that trace the data provenance and support results explainability. As workflows become more complex, tracing data provenance and explaining results become harder to achieve. In this paper, we propose a computational environment that automatically creates a workflow execution's record trail and invisibly attaches it to the workflow's output, enabling data traceability and results explainability. Our solution transforms existing container technology, includes tools for automatically annotating provenance metadata, and allows effective movement of data and metadata across the workflow execution. We demonstrate the capabilities of our environment with the study of SOMOSPIE, an earth science workflow. Through a suite of machine learning modeling techniques, this workflow predicts soil moisture values from the 27 km resolution satellite data down to higher resolutions necessary for policy making and precision agriculture. By running the workflow in our environment, we can identify the causes of different accuracy measurements for predicted soil moisture values in different resolutions of the input data and link different results to different machine learning methods used during the soil moisture downscaling, all without requiring scientists to know aspects of workflow design and implementation.

A Review of Containerization for Interactive and Reproducible Analysis

“…For example, code typically depends on programming language and operating system versions, and system-level library code (as in Figure 1). Recently, some have sought to solve these broader dependency issues using virtualization, a well-studied software engineering solution to dependency problems (Nüst et al 2020;Olaya et al 2020). Virtualization encapsulates code and all of its dependencies into a virtual computing environment that can be easily disseminated.…”

“…Containerization has been an increasingly adopted tool for reproducibility widely across the scientific community including areas such as geography, psychology, environmental science, metagenomics and many others (Knoth and Nüst 2017;Wiebels and Moreau 2021;Essawy et al 2020;Visconti et al 2018;Nüst et al 2020;Olaya et al 2020). To set the stage for a review of containerization technology we will first illustrate how containerization is used in practice.…”

Automatizing the creation of specialized high-performance computing containers

“…The use of containers is common in the area of HPC. For example, in Olaya et al (2022), the authors present an environment based on fine-grained containerization of both data and applications which automatically creates data lineage and record trail of workflow executions, enabling traceability of data and explainability of results. However, very few approaches to automating its deployment can be found in the literature.…”

Orchestration of materials science workflows for heterogeneous resources at large scale

“…There is a growing need for developing persistent scientific workflows to seamlessly connect and integrate software stacks and data services across cloud platforms supported by virtualization and data provenance (Bhatia et al, 2021). Containerization of scientific workflow enables reusability, portability, and reproducibility of results (Olaya et al, 2023) and ease of system maintenance efforts (Dusia et al, 2015); McDaniel et al, 2015); Monsalve et al, 2015); Herbein et al, 2016). Future directions point to the need for automatic containerization of complete working environments that include software dependencies (e.g., Python programs/modules) at all stages and the components of the Dask cluster, which are currently running natively in our workflow to maximize performance.…”