Discovering conditional inclusion dependencies

Abstract

Data dependencies are used to improve the quality of a database schema, to optimize queries, and to ensure consistency in a database. Conditional dependencies have been introduced to analyze and improve data quality. A conditional dependency is a dependency with a limited scope defined by conditions over one or more attributes. Only the matching part of the instance must adhere to the dependency. In this paper we focus on conditional inclusion dependencies (Cinds).We generalize the definition of Cinds, distinguishing covering and completeness conditions. We present a new use case for such Cinds showing their value for solving complex data quality tasks. Further, we propose efficient algorithms that identify covering and completeness conditions conforming to given quality thresholds. Our algorithms choose not only the condition values but also the condition attributes automatically. Finally, we show that our approach efficiently provides meaningful and helpful results for our use case.

Profiling relational data: a survey

“…If the distinct value sets of columns A and B are not available, we can estimate the Jaccard similarity using their MinHash signatures [38]. Conditional functional dependencies [24], [59], CTANE [47], CFUN [42], FACD [91], FastCFD [47] Inclusion dependencies [101], [87], SPIDER [14], ZigZag [102] Conditional inclusion dependencies [61], CINDERELLA [13], PLI [13] Foreign keys [123], [143] Denial constraints FastDC [29] Differential dependencies [128] Sequential dependencies [57] 5 Dependency detection…”

“…Similarly, Bauckmann et al [13] start with a set of approximate Inds and find pattern tuples to turn these into Cinds; however, in contrast to [61], they are not constrained to a single embedded Ind. The authors present two algorithms: CINDERELLA, which is based on the Apriori algorithm for association rule mining and employs a breadth-first traversal of the powerset lattice, and PLI, which employs a depth-first traversal instead.…”

Data Integration for Open Data on the Web

“…A core application for these dependencies is the discovery of foreign key relations across tables, but they are also used in data integration [53] scenarios, query optimization, and schema redesign [62]. The task of finding INDs gets harder with the number of tables and columns and the scalable and efficient discovery of inclusion dependencies across several tables is a well-known challenge in database research [9,62,43]. The state of the art research combines probabilistic and exact data structures to approximate the INDs in relational datasets.…”

Analysis of Typed Inclusion Dependencies with Null Values

“…Однако способствовать их построению позволяют алгоритмы, ко-торые определяют полноту условий за счет определения пороговых значений каче-ства [16,17].…”