Provenance trails in the Wings/Pegasus system

Kim, Jihie; Deelman, Ewa; Gil, Yolanda; Mehta, Gaurang; Ratnakar, Varun

doi:10.1002/cpe.1228

Cited by 106 publications

(75 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The provenance is generated by the middleware after inspection of the inputs and outputs of the wrapped components. This approach is popular with the scientific workflow community and includes systems such as Taverna [29], VisTrails [11], Kepler [2] and Wings [17]. Other middlewarebased systems like Karma [31] are not tied to a workflow system, but instead tap into the communication stack to capture provenance.…”

Section: Capturing Provenancementioning

confidence: 99%

Looking Inside the Black-Box: Capturing Data Provenance Using Dynamic Instrumentation

Stamatogiannakis¹,

Groth²,

Bos³

2015

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Knowing the provenance of a data item helps in ascertaining its trustworthiness. Various approaches have been proposed to track or infer data provenance. However, these approaches either treat an executing program as a black-box, limiting the fidelity of the captured provenance, or require developers to modify the program to make it provenance-aware. In this paper, we introduce DataTracker, a new approach to capturing data provenance based on taint tracking, a technique widely used in the security and reverse engineering fields. Our system is able to identify data provenance relations through dynamic instrumentation of unmodified binaries, without requiring access to, or knowledge of, their source code. Hence, we can track provenance for a variety of well-known applications. Because DataTracker looks inside the executing program, it captures high-fidelity and accurate data provenance.

show abstract

Section: Capturing Provenancementioning

confidence: 99%

Looking Inside the Black-Box: Capturing Data Provenance Using Dynamic Instrumentation

Stamatogiannakis¹,

Groth²,

Bos³

2015

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…The Wings-Pegasus system [64] uses an OWL ontology for semantic representation [65] of provenance generated during workflow instantiation and the Virtual Data System (VDS) provenance tracking catalog for provenance generated during workflow execution. As a result, workflow instantiation provenance can be queried using SPARQL and workflow execution provenance can be queried using SQL.…”

Section: Storing and Querying Scientific Workflow Provenancementioning

confidence: 99%

RDFProv: A relational RDF store for querying and managing scientific workflow provenance

Chebotko¹,

Lu²,

Fei³

et al. 2010

Data & Knowledge Engineering

View full text Add to dashboard Cite

Provenance metadata has become increasingly important to support scientific discovery reproducibility, result interpretation, and problem diagnosis in scientific workflow environments. The provenance management problem concerns the efficiency and effectiveness of the modeling, recording, representation, integration, storage, and querying of provenance metadata. Our approach to provenance management seamlessly integrates the interoperability, extensibility, and inference advantages of Semantic Web technologies with the storage and querying power of an RDBMS to meet the emerging requirements of scientific workflow provenance management. In this paper, we elaborate on the design of a relational RDF store, called RDFPROV, which is optimized for scientific workflow provenance querying and management. Specifically, we propose: i) two schema mapping algorithms to map an OWL provenance ontology to a relational database schema that is optimized for common provenance queries; ii) three efficient data mapping algorithms to map provenance RDF metadata to relational data according to the generated relational database schema, and iii) a schema-independent SPARQL-to-SQL translation algorithm that is optimized on-the-fly by using the type information of an instance available from the input provenance ontology and the statistics of the sizes of the tables in the database. Experimental results are presented to show that our algorithms are efficient and scalable. The comparison with two popular relational RDF stores, Jena and Sesame, and two commercial native RDF stores, AllegroGraph and BigOWLIM, showed that our optimizations result in improved performance and scalability for provenance metadata management. Finally, our case study for provenance management in a real-life biological simulation workflow showed the production quality and capability of the RDFPROV system. Although presented in the context of scientific workflow provenance management, many of our proposed techniques apply to general RDF data management as well.

show abstract

“…For example, we observe the use of provenance models in different workflow systems such as REDUX (Barga and Digiampietri 2008), Taverna, Pegasus (Kim et al 2008) and Karma (Simmhan et al 2008). These systems are based on two layers of provenance named retrospective layer-information about workflow executions-and prospective layer-information about workflow specifications (Clifford et al 2008).…”

Section: Ontologies To Support Shared Knowledgementioning

confidence: 99%

CI-Miner: semantically enhancing scientific processes

et al. 2009

View full text Add to dashboard Cite

The realization of an international cyberinfrastructure of shared resources to overcome time and space limitations is challenging scientists to rethink how to document their processes. Many known scientific process requirements that would normally be considered impossible to implement a few years ago are close to becoming a reality for scientists, such as large scale integration and data reuse, data sharing across distinct scientific domains, comprehensive support for explaining process results, and full search capability for scientific products across domains. This article introduces the CI-Miner approach that can be used to aggregate knowledge about scientific processes and their products through the use of semantic annotations. The article shows how this aggregated knowledge is used to benefit scientists during the development of their research activities. The discussion is grounded on lessons learned through the use of CI-Miner to semantically annotate scientific processes in the areas of geo-sciences,

show abstract

Provenance trails in the Wings/Pegasus system

Cited by 106 publications

References 10 publications

Looking Inside the Black-Box: Capturing Data Provenance Using Dynamic Instrumentation

Looking Inside the Black-Box: Capturing Data Provenance Using Dynamic Instrumentation

RDFProv: A relational RDF store for querying and managing scientific workflow provenance

CI-Miner: semantically enhancing scientific processes

Contact Info

Product

Resources

About