Supporting fine-grained data lineage in a database visualization environment

Woodruff, Allison; Stonebraker, Michael

doi:10.1109/icde.1997.581742

Cited by 122 publications

(97 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [WS97], a general framework is proposed for computing fine-grained data lineage in a transformational setting. The paper defines and traces data lineage for each transformation based on a weak inverse, which must be specified by the transformation definer.…”

Section: Related Workmentioning

confidence: 99%

Lineage tracing for general data warehouse transformations

Cui

Widom

2003

The VLDB Journal The International Journal on Very Large Data B

218

170

View full text Add to dashboard Cite

Data warehousing systems integrate information from operational data sources into a central repository to enable analysis and mining of the integrated information. During the integration process, source data typically undergoes a series of transformations, which may vary from simple algebraic operations or aggregations to complex "data cleansing" procedures. In a warehousing environment, the data lineage problem is that of tracing warehouse data items back to the original source items from which they were derived. We formally define the lineage tracing problem in the presence of general data warehouse transformations, and we present algorithms for lineage tracing in this environment. Our tracing procedures take advantage of known structure or properties of transformations when present, but also work in the absence of such information. Our results can be used as the basis for a lineage tracing tool in a general warehousing setting, and also can guide the design of data warehouses that enable efficient lineage tracing.

show abstract

Section: Related Workmentioning

confidence: 99%

Lineage tracing for general data warehouse transformations

Cui

Widom

2003

The VLDB Journal The International Journal on Very Large Data B

218

170

View full text Add to dashboard Cite

show abstract

“…Although provenance was studied earlier by Wang and Madnick [56], Woodruff and Stonebraker [59] and Cui et al [20], Buneman et al [9] has had greater influence (at least measured in terms of citations) than these other works. We conjecture that one reason for this is that Buneman et al went beyond proposing mechanisms for provenance: they also considered the question of the meaning of provenance.…”

Section: Introductionmentioning

confidence: 99%

Toward a Theory of Self-explaining Computation

Cheney

Acar

Perera

2013

In Search of Elegance in the Theory and Practice of Computation

View full text Add to dashboard Cite

Abstract. Provenance techniques aim to increase the reliability of human judgments about data by making its origin and derivation process explicit. Originally motivated by the needs of scientific databases and scientific computation, provenance has also become a major issue for business and government data on the Web. However, so far provenance has been studied only in relatively restrictive settings: typically, for data stored in databases or scientific workflow systems, and processed by query or workflow languages of limited expressiveness. Long-term provenance solutions require an understanding of provenance in other settings, particularly the general-purpose programming or scripting languages that are used to glue different components such as databases, Web services and workflows together. Moreover, what is required is not only an account of mechanisms for recording provenance, but also a theory of what it means for provenance information to explain or justify a computation. In this paper, we begin to outline a such a theory of self-explaining computation. We introduce a model of provenance for a simple imperative language based on operational derivations and explore its properties.

show abstract

“…As mentioned in the introduction, "workflow" or "coarse-grained" provenance has been studied extensively in the context of scientific computation [10,14,26]; Bose and Frew [3] and Simmhan et al [21] survey most existing research on such systems. These approaches record the process used to derive processed data products from raw data.…”

Section: Related Workmentioning

confidence: 99%

Provenance management in curated databases

Buneman

Chapman

Cheney

2006

Proceedings of the 2006 ACM SIGMOD International Conference on Management of Data

202

162

View full text Add to dashboard Cite

Curated databases in bioinformatics and other disciplines are the result of a great deal of manual annotation, correction and transfer of data from other sources. Provenance information concerning the creation, attribution, or version history of such data is crucial for assessing its integrity and scientific value. General purpose database systems provide little support for tracking provenance, especially when data moves among databases. This paper investigates general-purpose techniques for recording provenance for data that is copied among databases. We describe an approach in which we track the user's actions while browsing source databases and copying data into a curated database, in order to record the user's actions in a convenient, queryable form. We present an implementation of this technique and use it to evaluate the feasibility of database support for provenance management. Our experiments show that although the overhead of a naïve approach is fairly high, it can be decreased to an acceptable level using simple optimizations.

show abstract

Supporting fine-grained data lineage in a database visualization environment

Abstract: Abstract

Cited by 122 publications

References 9 publications

Lineage tracing for general data warehouse transformations

Lineage tracing for general data warehouse transformations

Toward a Theory of Self-explaining Computation

Provenance management in curated databases

Contact Info

Product

Resources

About