Complexity and Efficient Algorithms for Data Inconsistency Evaluating and Repairing

Abstract

Data inconsistency evaluating and repairing are major concerns in data quality management. As the basic computing task, optimal subset repair is not only applied for cost estimation during the progress of database repairing, but also directly used to derive the evaluation of database inconsistency. Computing an optimal subset repair is to find a minimum tuple set from an inconsistent database whose remove results in a consistent subset left. Tight bound on the complexity and efficient algorithms are still unknown. In this paper, we improve the existing complexity and algorithmic results, together with a fast estimation on the size of optimal subset repair. We first strengthen the dichotomy for optimal subset repair computation problem, we show that it is not only APXcomplete, but also NPhard to approximate an optimal subset repair with a factor better than 17/16 for most cases. We second show a (2 − 0.5 σ−1 )approximation whenever given σ functional dependencies, and a (2 − η k + η k k )-approximation when an η k -portion of tuples have the k-quasi-Turán property for some k > 1. We finally show a sublinear estimator on the size of optimal Srepair for subset queries, it outputs an estimation of a ratio 2n + n with a high probability, thus deriving an estimation of FD-inconsistency degree of a ratio 2 + . To support a variety of subset queries for FD-inconsistency evaluation, we unify them as the ⊆-oracle which can answer membership-query, and return p tuples uniformly sampled whenever given a number p. Experiments are conducted on * Supported by NSFC xxxx, NSFC xxxx range queries as an implementation of ⊆-oracle, and results show the efficiency of our FD-inconsistency degree estimator.

A Survey on the Role of Industrial IoT in Manufacturing for Implementation of Smart Industry

“…In addition, data inconsistency is usually compounded by redundancy. For example, multiple tables have the same data but different inputs [153].…”