Search for Computational Workflow Synergies in Reproducible Research Data Analyses in Particle Physics and Life Sciences

Šimko, Tibor; Cranmer, K.; Crusoe, Michael R.; Heinrich, L.; Khodak, Anton; Kousidis, Dinos; Rodríguez, D.

doi:10.1109/escience.2018.00123

Cited by 4 publications

(2 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…REANA was developed with particle physics use cases in mind, but the platform profits from synergies with general reproducible data analysis patterns in other scientific disciplines, such as astronomy, bioinformatics and life sciences [95].…”

Section: Reana Reproducible Analysesmentioning

confidence: 99%

Data Preservation in High Energy Physics (DPHEP Global Report 2022)

Basaglia¹

2023

View full text Add to dashboard Cite

This document summarizes the status of data preservation in high energy physics. The paradigms and the methodological advances are discussed from a perspective of more than ten years of experience with a structured effort at international level. The status and the scientific return related to the preservation of data accumulated at large collider experiments are presented, together with an account of ongoing efforts to ensure long-term analysis capabilities for ongoing and future experiments. Transverse projects aimed at generic solutions, most of which are specifically inspired by open science and FAIR principles, are presented as well. A prospective and an action plan are also indicated.

show abstract

Section: Reana Reproducible Analysesmentioning

confidence: 99%

Data Preservation in High Energy Physics (DPHEP Global Report 2022)

Basaglia¹

2023

View full text Add to dashboard Cite

show abstract

“…The REANA framework [1], used by the SCAILFIN project [2] to create analysis workflows that are easily reproducible, works on top of Kubernetes in order to orchestrate all service components and the scheduling of workers to run the payloads. This assumes support of docker and a shared file system between services and workers (see Figure 1).…”

Section: Introductionmentioning

confidence: 99%

Abstracting container technologies and transfer mechanisms in the Scalable CyberInfrastructure for Artificial Intelligence and Likelihood Free Inference (SCAILFIN) project

et al. 2020

Self Cite

View full text Add to dashboard Cite

High Performance Computing (HPC) facilities provide vast computational power and storage, but generally work on fixed environments designed to address the most common software needs locally, making it challenging for users to bring their own software. To overcome this issue, most HPC facilities have added support for HPC friendly container technologies such as Shifter, Singularity, or Charliecloud. These different container technologies are all compatible with the more popular Docker containers, however the implementation and use of said containers is different for each HPC friendly container technology. These usage differences can make it difficult for an end user to easily submit and utilize different HPC sites without making adjustments to their workflows and software. This issue is exacerbated when attempting to utilize workflow management software between different sites with differing container technologies. The SCAILFIN project aims to develop and deploy artificial intelligence (AI) and likelihood-free inference (LFI) techniques and software using scalable cyberinfrastructure (CI) that span multiple sites. The project has extended the CERN-based REANA framework, a platform designed to enable analysis reusability, and reproducibility while supporting different workflow engine languages, in order to support submission to different HPC facilities. The work presented here focuses on the development of an abstraction layer that allows the support of different container technologies and different transfer protocols for files and directories between the HPC facility and the REANA cluster edge service from the user’s workflow application.

show abstract