An Annotation Management System for Relational Databases

Abstract

We present an annotation management system for relational databases. In this system, every piece of data in a relation is assumed to have zero or more annotations associated with it and annotations are propagated along, from the source to the output, as data is being transformed through a query. Such an annotation management system is important for understanding the provenance and quality of data, especially in applications that deal with integration of scientific and biological data.We present an extension, pSQL, of a fragment of SQL that has three different types of annotation propagation schemes, each useful for different purposes. The default scheme propagates annotations according to where data is copied from. The default-all scheme propagates annotations according to where data is copied from among all equivalent formulations of a given query. The custom scheme allows a user to specify how annotations should propagate. We present a storage scheme for the annotations and describe algorithms for translating a pSQL query under each propagation scheme into one or more SQL queries that would correctly retrieve the relevant annotations according to the specified propagation scheme. For the default-all scheme, we also show how we generate finitely many queries that can simulate the annotation propagation behavior of the set of all equivalent queries, which is possibly infinite. The algorithms are implemented and the feasibility of the system is demonstrated by a set of experiments that we have conducted.

On the Complexity of View Update Analysis and Its Application to Annotation Propagation

“…With this comes the need for studying annotation propagation. The analysis of annotation propagation is important in tracing the origin of the data [1], [2], [5], [6], [7], [8] (a.k.a. lineage [9], [10]), data cleaning [11], access control [12], semantic web [13], and in digital libraries [14], among other things.…”

“…Recent research on querying annotated databases could be classified into two categories: annotation querying (e.g., [24], [25]) and annotation propagation (e.g., [1], [6], [8], [26], [27], [28]). In the former, queries access annotations as well as the regular data directly.…”

“…More specifically, propagation schemes for processing annotations explicitly or implicitly were studies in [8], and the expressive power of various propagation schemes was investigated in [25], [26]. The problems studied there are entirely different from the problems considered in this work.…”

“…The problems studied there are entirely different from the problems considered in this work. Other work investigated either propagating annotations of different granularity [6], or via different methods such as semirings [27], original source tags [28], and source set [8]. The only previous complexity results on annotation propagation were established in [1], [12].…”

LabelFlow: Exploiting Workflow Provenance to Surface Scientific Data Provenance

“…These approaches propose white-box workflow activities that correspond to relational query operators. The benefit of white-box steps is that they allow full-transparency and enable fine-grained lineage, also making way for the tracking of cell-level value-copying and annotation propagation [BC+04]. Similarly, work on dependency analysis in programming languages has recently found applicability as a formal foundation for the tracking of Nested Relational Calculus query provenance [CAA07].…”