Analyses and Validation of Conditional Dependencies with Built-in Predicates

Abstract

Abstract. This paper proposes a natural extension of conditional functional dependencies (cfds [14]) and conditional inclusion dependencies (cinds [8]), denoted by cfd p s and cind p s, respectively, by specifying patterns of data values with =, <, ≤, > and ≥ predicates. As data quality rules, cfd p s and cind p s are able to capture errors that commonly arise in practice but cannot be detected by cfds and cinds. We establish two sets of results for central technical problems associated with cfd p s and cind p s. (a) One concerns the satisfiability and implication problems for cfd p s and cind p s, taken separately or together. These are important for, e.g., deciding whether data quality rules are dirty themselves, and for removing redundant rules. We show that despite the increased expressive power, the static analyses of cfd p s and cind p s retain the same complexity as their cfds and cinds counterparts. (b) The other concerns validation of cfd p s and cind p s. We show that given a set Σ of cfd p s and cind p s on a database D, a set of sql queries can be automatically generated that, when evaluated against D, return all tuples in D that violate some dependencies in Σ. This provides commercial dbms with an immediate capability to detect errors based on cfd p s and cind p s.

Extending Conditional Dependencies with Built-in Predicates

“…Related work. This paper is an extension of our earlier work [12] by adding (a) the proofs for the complexity bounds for the satisfiability and implication analyses of CFD p s and CIND p s, separately and taken together (Section 4), and (b) an extensive experimental study of CFD p s and CIND p s (Section 6), which was not investigated in [12].…”

“…While there have been extensions of CFDs [10], [13], [28], none of these allows dependencies to be specified with patterns on data values in terms of built-in predicates ̸ =, <, ≤, > or ≥. To the best of our knowledge, the earlier conference version [12] of this paper is the first to study these constraints.…”

Discovery of Field Functional Dependencies

“…Inconsistency is one of the most important aspects of dirty data. A database is inconsistent if it violate some data quality rules such as functional dependencies [11], conditional functional dependencies [12], [13](CFDs), extended conditional functional dependencies [14], editing rules [15] and fixing rules [16] Dept], and the two dependencies above will never be violated by r.…”

“…To this end, Fan et al extended FDs in [12], [13] More over, a CFD can assign at most one constant value on each attribute, result in limited expressivity. To this end, more type of constrains were proposed to detect and repair data errors more thoroughly, including eCFDs [16], CFD p s [15], editing rules [17], fixing rules [18] and differential dependencies [19], comparable dependencies [20], etc.…”

Discovering conditional inclusion dependencies

“…Research has focussed on two types, i. e., conditional functional dependencies (Cfds) [4] and conditional inclusion dependencies (Cinds) [5]. Results have been published on reasoning about consistency and implication [5,6], validation of known conditional dependencies [6], or detection of conditional functional dependencies [11,12].…”