The LLUNATIC data-cleaning framework

Abstract

Data-cleaning (or data-repairing) is considered a crucial problem in many database-related tasks. It consists in making a database consistent with respect to a set of given constraints. In recent years, repairing methods have been proposed for several classes of constraints. However, these methods rely on ad hoc decisions and tend to hard-code the strategy to repair conflicting values. As a consequence, there is currently no general algorithm to solve database repairing problems that involve different kinds of constraints and different strategies to select preferred values. In this paper we develop a uniform framework to solve this problem. We propose a new semantics for repairs, and a chase-based algorithm to compute minimal solutions. We implemented the framework in a DBMSbased prototype, and we report experimental results that confirm its good scalability and superior quality in computing repairs.

Bleach: A Distributed Stream Data Cleaning System

“…Many approaches [6,20,4,9,10,19,16] tackle the problem of detecting and repairing dirty data based on predefined data quality rules. [9] proposes a way to combine multiple rules together and to perform data cleaning work holistically.…”

Descriptive and prescriptive data cleaning

“…Several algorithms have been proposed for repairing inconsistent data, mainly based on declarative data quality rules (such as in Σ D ) [1,10,3]. These rules naturally have a static semantics for violation detection (as described above) and a dynamic semantics to remove them.…”

“…Data cleaning focuses on detecting and repairing errors on a database by using declarative constraints [3,15,7,10,1]. In our target Error module, we can use any of these algorithms.…”

“…A common approach to address the problem of dirty data [8] is to apply a set of data quality rules or constraints over a target database, to "detect" and to eventually "repair" erroneous data [1,7,10,3,15]. Tuples or cells (attributevalue of a tuple) in a database D that are inconsistent w.r.t.…”

Partial results in database systems

“…In every instance of an incomplete input, our traditional database instincts tell us that the solution is to fix the problem: we should either replicate the data sources comprising the distributed system or make them more reliable; we should add replication and failover to the nodes of our parallel DBMS; or we should embark on data cleaning and repairing efforts to fix the incomplete tables and views in the single node system [4,5,7,10,14,16,20,30,[33][34][35][36]. However, these solutions can be either financially costly, performance hindering, or both.…”