Data cleaning and query answering with matching dependencies and matching functions

Bertossi, Leopoldo; Kolahi, Solmaz; Lakshmanan, Laks V. S.

doi:10.1145/1938551.1938585

Cited by 59 publications

(95 citation statements)

References 45 publications

(67 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The same holds for mapping & cleaning scenarios. Furthermore, some of the ingredients of our scenarios are inspired by, but different from, features of other repairing approaches (e.g., repairing based on both premise and conclusion of constraints [18], [10], cells [10], [9], groups of cells [9], partial orders and its incorporation in the chase [21]). As previously observed, these approaches support limited classes of constraints.…”

Section: Related Workmentioning

confidence: 99%

Mapping and cleaning

Geerts

Mecca

Papotti

et al. 2014

2014 IEEE 30th International Conference on Data Engineering

View full text Add to dashboard Cite

We address the challenging and open problem of bringing together two crucial activities in data integration and data quality, i.e., transforming data using schema mappings, and fixing conflicts and inconsistencies using data repairing. This problem is made complex by several factors. First, schema mappings and data repairing have traditionally been considered as separate activities, and research has progressed in a largely independent way in the two fields. Second, the elegant formalizations and the algorithms that have been proposed for both tasks have had mixed fortune in scaling to large databases. In the paper, we introduce a very general notion of a mapping and cleaning scenario that incorporates a wide variety of features, like, for example, user interventions. We develop a new semantics for these scenarios that represents a conservative extension of previous semantics for schema mappings and data repairing. Based on the semantics, we introduce a chase-based algorithm to compute solutions. Appropriate care is devoted to developing a scalable implementation of the chase algorithm. To the best of our knowledge, this is the first general and scalable proposal in this direction.

show abstract

Section: Related Workmentioning

confidence: 99%

Mapping and cleaning

Geerts

Mecca

Papotti

et al. 2014

2014 IEEE 30th International Conference on Data Engineering

View full text Add to dashboard Cite

show abstract

“…A number of recent research [4,7,10] have investigated the data cleaning problem introduced in [2]: repairing is to find another database that is consistent and minimally differs from the original database. They compute a consistent database by using di↵erent cost functions for value updates ⇤ Work done while interning at QCRI.…”

Section: Introductionmentioning

confidence: 99%

Towards dependable data repairing with fixing rules

Wang

Tang

2014

Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data

View full text Add to dashboard Cite

One of the main challenges that data cleaning systems face is to automatically identify and repair data errors in a dependable manner. Though data dependencies (a.k.a. integrity constraints) have been widely studied to capture errors in data, automated and dependable data repairing on these errors has remained a notoriously hard problem. In this work, we introduce an automated approach for dependably repairing data errors, based on a novel class of fixing rules. A fixing rule contains an evidence pattern, a set of negative patterns, and a fact value. The heart of fixing rules is deterministic: given a tuple, the evidence pattern and the negative patterns of a fixing rule are combined to precisely capture which attribute is wrong, and the fact indicates how to correct this error. We study several fundamental problems associated with fixing rules, and establish their complexity. We develop e cient algorithms to check whether a set of fixing rules is consistent, and discuss approaches to resolve inconsistent fixing rules. We also devise e cient algorithms for repairing data errors using fixing rules. We experimentally demonstrate that our techniques outperform other automated algorithms in terms of the accuracy of repairing data errors, using both real-life and synthetic data.

show abstract

“…Specifically, state of the art approaches (e.g., [9], [10], [11]) attempt to clean data by exploiting patterns in the data, which they express in the form of conditional functional dependencies (or CFDs). However, these approaches depend on the availability of a clean data corpus or an external reference table to learn data quality rules or patterns before fixing the errors in the dirty data.…”

Section: Introductionmentioning

confidence: 99%

BayesWipe: A multimodal system for data cleaning and consistent query answering on structured bigdata

Chen

et al. 2014

2014 IEEE International Conference on Big Data (Big Data)

View full text Add to dashboard Cite

Recent efforts in data cleaning of structured data have focused exclusively on problems like data deduplication, record matching, and data standardization; none of these focus on fixing incorrect attribute values in tuples. Correcting values in tuples is typically performed by a minimum cost repair of tuples that violate static constraints like CFDs (which have to be provided by domain experts, or learned from a clean sample of the database). In this paper, we provide a method for correcting individual attribute values in a structured database using a Bayesian generative model and a statistical error model learned from the noisy database directly. We thus avoid the necessity for a domain expert or clean master data. We also show how to efficiently perform consistent query answering using this model over a dirty database, in case write permissions to the database are unavailable. We evaluate our methods over both synthetic and real data.

show abstract

Data cleaning and query answering with matching dependencies and matching functions

Cited by 59 publications

References 45 publications

Mapping and cleaning

Mapping and cleaning

Towards dependable data repairing with fixing rules

BayesWipe: A multimodal system for data cleaning and consistent query answering on structured bigdata

Contact Info

Product

Resources

About