Provenance-based analysis of data-centric processes

Deutch, Daniel; Moskovitch, Yuval; Tannen, Val

doi:10.1007/s00778-015-0390-5

Cited by 15 publications

(13 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A data-dependent process (DDP) is a finite state machine that evaluates queries against a relational database to determine some transitions and uses external requests to trigger other transitions [9,10]. In [9] and [10], Deutch et al extend provenance semirings [13] to linear temporal logic formulas issued against a DDP. Both DDP provenance and wat-provenance aim to extend traditional data provenance to state machines, but they do so in very different ways.…”

Section: Related Workmentioning

confidence: 99%

Debugging Distributed Systems with Why-Across-Time Provenance

Whittaker

Teodoropol

Alvaro

et al. 2018

Proceedings of the ACM Symposium on Cloud Computing

View full text Add to dashboard Cite

Systematically reasoning about the fine-grained causes of events in a real-world distributed system is challenging. Causality, from the distributed systems literature, can be used to compute the causal history of an arbitrary event in a distributed system, but the event's causal history is an overapproximation of the true causes. Data provenance, from the database literature, precisely describes why a particular tuple appears in the output of a relational query, but data provenance is limited to the domain of static relational databases. In this paper, we present wat-provenance: a novel form of provenance that provides the benefits of causality and data provenance. Given an arbitrary state machine, watprovenance describes why the state machine produces a particular output when given a particular input. This enables system developers to reason about the causes of events in real-world distributed systems. We observe that automatically extracting the wat-provenance of a state machine is often infeasible. Fortunately, many distributed systems components have simple interfaces from which a developer can directly specify wat-provenance using a technique we call wat-provenance specifications. Leveraging the theoretical foundations of wat-provenance, we implement a prototype distributed debugging framework called Watermelon.Returning to our example above, the why-provenance of the (Michael Jordan) tuple is the set {J 1 , J 2 } where J 1 = {(ecodd, jumpman), (jumpman, Michael Jordan)} J 2 = {(ecodd, mlpro), (mlpro, Michael Jordan)} J 1 and J 2 are minimal witnesses, so the why-provenance and minimal why-provenance of t = (Michael Jordan) are the same.

show abstract

Section: Related Workmentioning

confidence: 99%

Debugging Distributed Systems with Why-Across-Time Provenance

Whittaker

Teodoropol

Alvaro

et al. 2018

Proceedings of the ACM Symposium on Cloud Computing

View full text Add to dashboard Cite

show abstract

“…We have implemented our provenance framework in the context of PROPOLIS (PROvisioned Process Analysis) [13]. The system architecture is shown in Fig.…”

Section: Prototype Implementationmentioning

confidence: 99%

“…We have implemented our model and algorithms in the context of a system prototype called PROPOLIS (PROvisioned data-dependent PrOcess anaLysIS) [13]. PROPOLIS allows analysts to define, in addition to an LTL formula, annotations (from e.g.…”

Section: Introductionmentioning

confidence: 99%

A provenance framework for data-dependent process analysis

2014

Self Cite

View full text Add to dashboard Cite

A data-dependent process (DDP) models an application whose control flow is guided by a finite state machine, as well as by the state of an underlying database. DDPs are commonly found e.g., in e-commerce. In this paper we develop a framework supporting the use of provenance in static (temporal) analysis of possible DDP executions. Using provenance support, analysts can interactively test and explore the effect of hypothetical modifications to a DDP's state machine and/or to the underlying database. They can also extend the analysis to incorporate the propagation of annotations from meta-domains of interest, e.g., cost or access privileges.Toward this goal we note that the framework of semiringbased provenance was proven highly effective in fulfilling similar needs in the context of database queries. In this paper we consider novel constructions that generalize the semiring approach to the context of DDP analysis. These constructions address two interacting new challenges: (1) to combine provenance annotations for both information that resides in the database and information about external inputs (e.g., user choices), and (2) to finitely capture infinite process executions. We analyze our solution from theoretical and experimental perspectives, proving its effectiveness.

show abstract

“…In a nutshell, data provenance captures details of the computation that took place and resulted in the generation of each output data item. Multiple models for data provenance have been proposed, for multiple query languages such as the (positive) relational algebra, datalog (see [23]), data-intensive workflows (e.g., [14,35]) data mining [21], and datacentric applications [16]. Provenance has been proven useful for managing access control, trust, hypothetical reasoning, view maintenance and debugging (see [19,22,15,23,8]).…”

Section: Introductionmentioning

confidence: 99%

Equivalence-Invariant Algebraic Provenance for Hyperplane Update Queries

Bourhis¹,

Deutch

Moskovitch

2020

Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data

Self Cite

View full text Add to dashboard Cite

The algebraic approach for provenance tracking, originating in the semiring model of Green et. al, has proven useful as an abstract way of handling metadata. Commutative Semirings were shown to be the "correct" algebraic structure for Union of Conjunctive Queries, in the sense that its use allows provenance to be invariant under certain expected query equivalence axioms. In this paper we present the first (to our knowledge) algebraic provenance model, for a fragment of update queries, that is invariant under set equivalence. The fragment that we focus on is that of hyperplane queries, previously studied in multiple lines of work. Our algebraic provenance structure and corresponding provenance-aware semantics are based on the sound and complete axiomatization of Karabeg and Vianu. We demonstrate that our construction can guide the design of concrete provenance model instances for different applications. We further study the efficient generation and storage of provenance for hyperplane update queries. We show that a naive algorithm can lead to an exponentially large provenance expression, but remedy this by presenting a normal form which we show may be efficiently computed alongside query evaluation. We experimentally study the performance of our solution and demonstrate its scalability and usefulness, and in particular the effectiveness of our normal form representation.

show abstract

Provenance-based analysis of data-centric processes

Cited by 15 publications

References 29 publications

Debugging Distributed Systems with Why-Across-Time Provenance

Debugging Distributed Systems with Why-Across-Time Provenance

A provenance framework for data-dependent process analysis

Equivalence-Invariant Algebraic Provenance for Hyperplane Update Queries

Contact Info

Product

Resources

About