A Core Calculus for Provenance

Abstract

Abstract.Provenance is an increasing concern due to the revolution in sharing and processing scientific data on the Web and in other computer systems. It is proposed that many computer systems will need to become provenance-aware in order to provide satisfactory accountability, reproducibility, and trust for scientific or other high-value data. To date, there is not a consensus concerning appropriate formal models or security properties for provenance. In previous work, we introduced a formal framework for provenance security and proposed formal definitions of properties called disclosure and obfuscation. This paper develops a core calculus for provenance in programming languages. Whereas previous models of provenance have focused on special-purpose languages such as workflows and database queries, we consider a higher-order, functional language with sums, products, and recursive types and functions. We explore the ramifications of using traces based on operational derivations for the purpose of comparing other forms of provenance. We design a rich class of provenance views over traces. Finally, we prove relationships among provenance views and develop some solutions to the disclosure and obfuscation problems.

Database Queries that Explain their Work

“…The main technical contribution of this paper over our previous work [1,27] is its treatment of tracing and slicing for collections. There are two underlying technical challenges; we illustrate both (and our solutions) via a simple example query Q = σA<B(R) ∪ ρA →B,B →A(σA≥B (R)) over a table R with attributes A, B.…”

“…The main new ideas are in the rules for collection operations, particularly comprehensions. The slicing rules SCONST, SPRIM, SVAR, SLET, SPAIR, SPROJi, and SIF follow essentially the same idea as in our previous work [1,27]. We focus discussion on the new cases, but we review the key ideas for these operations here in order to make the presentation selfcontained.…”

“…The rule SEMPTY is essentially the same as the constant rule. The rule SSNG uses the singleton extraction operation p.ǫ to obtain a pattern describing the single element of a singleton set value matching p. The rule SUNION uses the two projections p [1] and p [2] to obtain the patterns describing the subsets obtained from the first and second subtraces in the union pattern, respectively.…”

“…A natural question is whether slicing computes the (or a) smallest possible answer. Because our definition of correct slices is based on recomputation, minimal slices are not computable, by a straightforward reduction from the undecidability of minimizing dependency provenance [1,12].…”

“…More recently, we proposed an approach to provenance called self-explaining computation [11] and explored it in the context of a general-purpose functional programming language [1,27]. In this approach, detailed execution traces are used as a form of provenance.…”