The End-to-End Provenance Project

Ellison, Aaron M.; Boose, Emery R.; Lerner, Barbara; Fong, Elizabeth; Seltzer, Margo

doi:10.1016/j.patter.2020.100016

Cited by 4 publications

(4 citation statements)

References 7 publications

(8 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent end-to-end provenance projects have developed a set of tools, such as R packages, that allow organizations to establish data provenance through enhanced data traceability [24].…”

Section: Demanding Data Traceabilitymentioning

confidence: 99%

Establishing Data Provenance for Responsible Artificial Intelligence Systems

Werder

Ramesh

Zhang

2022

ACM Trans. Manage. Inf. Syst.

View full text Add to dashboard Cite

Data provenance, a record that describes the origins and processing of data, offers new promises in the increasingly important role of artificial intelligence (AI)-based systems in guiding human decision making. To avoid disastrous outcomes that can result from bias-laden AI systems, responsible AI builds on four important characteristics: fairness, accountability, transparency, and explainability. To stimulate further research on data provenance that enables responsible AI, this study outlines existing biases and discusses possible implementations of data provenance to mitigate them. We first review biases stemming from the data's origins and pre-processing. We then discuss the current state of practice, the challenges it presents, and corresponding recommendations to address them. We present a summary highlighting how our recommendations can help establish data provenance and thereby mitigate biases stemming from the data's origins and pre-processing to realize responsible AI-based systems. We conclude with a research agenda suggesting further research avenues.

show abstract

“…Recent end-to-end provenance projects have developed a set of tools, such as R packages, that allow organizations to establish data provenance through enhanced data traceability [24].…”

Section: Demanding Data Traceabilitymentioning

confidence: 99%

Establishing Data Provenance for Responsible Artificial Intelligence Systems

Werder

Ramesh

Zhang

2022

ACM Trans. Manage. Inf. Syst.

View full text Add to dashboard Cite

show abstract

“…Language-specific solutions, such as noworkflow [57] for python scripts, or rdt for R scripts among others have been developed and offer richer support such as visualisation [20]. Prov-O serves as a fundamental building block for capturing provenance-related data.…”

Section: Fj Ekaputra Et Al / Semantic-enabled Architecture For Auditable Privacy-preserving Data Analysismentioning

confidence: 99%

Semantic-enabled architecture for auditable privacy-preserving data analysis

Ekaputra

Ekelhart

Mayer

et al. 2024

View full text Add to dashboard Cite

Small and medium-sized organisations face challenges in acquiring, storing and analysing personal data, particularly sensitive data (e.g., data of medical nature), due to data protection regulations, such as the GDPR in the EU, which stipulates high standards in data protection. Consequently, these organisations often refrain from collecting data centrally, which means losing the potential of data analytics and learning from aggregated user data. To enable organisations to leverage the full-potential of the collected personal data, two main technical challenges need to be addressed: (i) organisations must preserve the privacy of individual users and honour their consent, while (ii) being able to provide data and algorithmic governance, e.g., in the form of audit trails, to increase trust in the result and support reproducibility of the data analysis tasks performed on the collected data. Such an auditable, privacy-preserving data analysis is currently challenging to achieve, as existing methods and tools only offer partial solutions to this problem, e.g., data representation of audit trails and user consent, automatic checking of usage policies or data anonymisation. To the best of our knowledge, there exists no approach providing an integrated architecture for auditable, privacy-preserving data analysis. To address these gaps, as the main contribution of this paper, we propose the WellFort approach, a semantic-enabled architecture for auditable, privacy-preserving data analysis which provides secure storage for users’ sensitive data with explicit consent, and delivers a trusted, auditable analysis environment for executing data analytic processes in a privacy-preserving manner. Additional contributions include the adaptation of Semantic Web technologies as an integral part of the WellFort architecture, and the demonstration of the approach through a feasibility study with a prototype supporting use cases from the medical domain. Our evaluation shows that WellFort enables privacy preserving analysis of data, and collects sufficient information in an automated way to support its auditability at the same time.

show abstract

“…Works focusing on provenance per se such as (Alterovitz et al, 2018;Ellison et al, 2020) and the various workflow provenance systems such as (Khan et al, 2019;Papadimitriou et al, 2021;Yakutovich et al, 2021) are primarily concerned with very detailed documentation of each computation on one or more datasets. The W3C PROV model (Gil et al, 2013;Lebo et al, 2013;Moreau et al, 2013) was developed initially to support interoperability across the transformation logs of workflow systems.…”

Section: Related Workmentioning

confidence: 99%

FAIRSCAPE: a Framework for FAIR and Reproducible Biomedical Analytics

et al. 2021

View full text Add to dashboard Cite

Results of computational analyses require transparent disclosure of their supporting resources, while the analyses themselves often can be very large scale and involve multiple processing steps separated in time. Evidence for the correctness of any analysis should include not only a textual description, but also a formal record of the computations which produced the result, including accessible data and software with runtime parameters, environment, and personnel involved. This article describes FAIRSCAPE, a reusable computational framework, enabling simplified access to modern scalable cloud-based components. FAIRSCAPE fully implements the FAIR data principles and extends them to provide fully FAIR Evidence, including machine-interpretable provenance of datasets, software and computations, as metadata for all computed results. The FAIRSCAPE microservices framework creates a complete Evidence Graph for every computational result, including persistent identifiers with metadata, resolvable to the software, computations, and datasets used in the computation; and stores a URI to the root of the graph in the result’s metadata. An ontology for Evidence Graphs, EVI (https://w3id.org/EVI), supports inferential reasoning over the evidence. FAIRSCAPE can run nested or disjoint workflows and preserves provenance across them. It can run Apache Spark jobs, scripts, workflows, or user-supplied containers. All objects are assigned persistent IDs, including software. All results are annotated with FAIR metadata using the evidence graph model for access, validation, reproducibility, and re-use of archived data and software.

show abstract

The End-to-End Provenance Project

Cited by 4 publications

References 7 publications

Establishing Data Provenance for Responsible Artificial Intelligence Systems

Establishing Data Provenance for Responsible Artificial Intelligence Systems

Semantic-enabled architecture for auditable privacy-preserving data analysis

FAIRSCAPE: a Framework for FAIR and Reproducible Biomedical Analytics

Contact Info

Product

Resources

About