Automated metadata, provenance cataloging and navigable interfaces: Ensuring the usefulness of extreme-scale data

Schissel, D.P.; Abla, G.; Flanagan, S.; Greenwald, M.; Lee, X.; Romosan, Alex; Shoshani, Arie; Stillerman, J.; Wright, J. C.

doi:10.1016/j.fusengdes.2014.01.053

Cited by 12 publications

(8 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The first production version of the MPO system architecture is slightly different than the prototype version reported earlier [2]. In the prototype version, both the API service and the interactive UI service were provided by a single web server and they used the same network port.…”

Section: The Mpo Architecturementioning

confidence: 99%

See 1 more Smart Citation

The MPO system for automatic workflow documentation

Abla

Coviello

Flanagan

et al. 2016

Fusion Engineering and Design

Self Cite

View full text Add to dashboard Cite

Data from large-scale experiments and extreme-scale computing is expensive to produce and may be used for critical applications. However, it is not the mere existence of data that is important, but our ability to make use of it. Experience has shown that when metadata is better organized and more complete, the underlying data becomes more useful. Traditionally, capturing the steps of scientific workflows and metadata was the role of the lab notebook, but the digital era has resulted instead in the fragmentation of data, processing, and annotation. This paper presents the Metadata, Provenance, and Ontology (MPO) System, the software that can automate the documentation of scientific workflows and associated information. Based on recorded metadata, it provides explicit information about the relationships among the elements of workflows in notebook form augmented with directed acyclic graphs. A set of web-based graphical navigation tools and Application Programming Interface (API) have been created for searching and browsing, as well as programmatically accessing the workflows and data. We describe the MPO concepts and its software architecture. We also report the current status of the software as well as the initial deployment experience.

show abstract

Section: The Mpo Architecturementioning

confidence: 99%

“…The motivation of the project and design principles of the prototype system have been described earlier [1,2,3]. This paper discusses the most recent results of the MPO project's research, including the current status of the MPO software system and deployment experiences.…”

Section: Introductionmentioning

confidence: 99%

The MPO system for automatic workflow documentation

Abla

Coviello

Flanagan

et al. 2016

Fusion Engineering and Design

Self Cite

View full text Add to dashboard Cite

show abstract

“…Knowledge about relevant metadata is a baseline requirement to describe, evaluate und master the remaining uncertainty in data management. Especially in collaborative research, uniform and software independent data access is needed to provide equal benefit for all researches and participants [8] [9]. The goal is to manage datasets by capturing metadata in a uniform data structure and to centralize its access to improve the overall level of documentation, to allow the traceability of results and to reduce the barriers associated with collaboratively used data.…”

Section: Introductionmentioning

confidence: 99%

Methods and Technologies for Research- and Metadata Management in Collaborative Experimental Research

Preuß

Staudter

Weber

et al. 2018

AMM

View full text Add to dashboard Cite

Newly developed technologies and methods for the purpose of controlling uncertainty in technical systems must be proven and validated against reliable experimental studies. The availability of descriptive metadata is mandatory to enable long term usability and sharing of such experimental research data. This article introduces a concept for a software independent solution for managing data in collaborative research environments. The proposed approach leverages the advantages of capturing metadata in a uniform, modular data structure and providing software independent access to a centralized data repository as well as its contents by means of a web-application. The article presents a prototype implementation of the proposed approach and discusses its application on the demonstrator test rig of a collaborative research centre in the field of mechanical engineering.

show abstract

“…), enabling the reproduction of code execution, from data preparation to data exporting and publication. The content of the catalogue, based on the W7 [8,9] model of provenance, corresponds to answers to the questions "Who, What, When, How and Why" about resources and activities involved in producing the final outputs of the code execution. Provenance Metadata Gathering System can be thought of as the back-end of an electronic notebook for modeling and data analysis.…”

Section: Introductionmentioning

confidence: 99%