Revisiting Conditional Functional Dependency Discovery: Splitting the “C” from the “FD”

Rammelaere, Joeri; Geerts, Floris

doi:10.1007/978-3-030-10928-8_33

Cited by 11 publications

(5 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Integrity Constraints Discovery. Due to the importance of ICs, many techniques and systems have been proposed for discovering different kinds of ICs, namely, FDs [26,30,43], temporal FDs [2], differential dependencies [38], conditional differential dependencies [25], CFDs [14,35], and order dependencies [40]. These techniques work on entire values and adapting them to PFD discovery is quite challenging; we have to carefully combine pattern discovery and data dependency discovery, to effectively and efficiently discover PFDs.…”

Section: Related Workmentioning

confidence: 99%

Pattern functional dependencies for data cleaning

Qahtan

Tang²,

Ouzzani³

et al. 2020

Proc. VLDB Endow.

View full text Add to dashboard Cite

Patterns (or regex-based expressions) are widely used to constrain the format of a domain (or a column), e.g., a Year column should contain only four digits, and thus a value like "1980-" might be a typo. Moreover, integrity constraints (ICs) defined over multiple columns, such as (conditional) functional dependencies and denial constraints, e.g., a ZIP code uniquely determines a city in the UK, have been widely used in data cleaning. However, a promising, but not yet explored, direction is to combine regex-and IC-based theories to capture data dependencies involving partial attribute values. For example, in an employee ID such as"F-9-107", "F" is sufficient to determine the finance department. Inspired by the above observation, we propose a novel class of ICs, called pattern functional dependencies (PFDs), to model fine-grained data dependencies gleaned from partial attribute values. These dependencies cannot be modeled using traditional ICs, such as (conditional) functional dependencies, which work on entire attribute values. We also present a set of axioms for the inference of PFDs, analogous to Armstrong's axioms for FDs, and study the complexity of consistency and implication analysis of PFDs. Moreover, we devise an effective algorithm to automatically discover PFDs even in the presence of errors in the data. Our extensive experiments on 15 real-world datasets show that our approach can effectively discover valid and useful PFDs over dirty data, which can then be used to detect data errors that are hard to capture by other types of ICs.

show abstract

Section: Related Workmentioning

confidence: 99%

Pattern functional dependencies for data cleaning

Qahtan

Tang²,

Ouzzani³

et al. 2020

Proc. VLDB Endow.

View full text Add to dashboard Cite

show abstract

“…Discovering data dependencies is an active and long-standing research problem within the database and data mining communities. Indeed, the problem is well-studied for the relational data setting, with volumes of contributions for functional dependencies (FDs) [8,19,44,18] and its numerous extensions (e.g., conditional FDs [7,26,24], distance-based FDs [10,9,14,15,13], etc).…”

Section: Introductionmentioning

confidence: 99%

An Efficient Approach for Discovering Graph Entity Dependencies (GEDs)

Liu¹,

Kwashie²,

Zhang³

et al. 2023

Preprint

View full text Add to dashboard Cite

A study of the discovery problem of Graph Entity Dependencies (GEDs).• A new and efficient approach for the discovery of GEDs in property graphs.• A minimum description length inspired definition of interestingness of GEDs to rank discovered rules.• A thorough empirical evaluation of the proposed technique, with examples of useful rules (mined) that are relevant in data quality/management applications.

show abstract

“…Integrity constraints may be obtained by domain experts; however, this is often an expensive task that requires expertise not only in the domain but also in the constraint language. In the past two decades, extensive effort has been invested in exploring the challenge of automatically discovering constraints from the data itself, for different types of constraints, including the classic Functional Dependencies (FDs) [15,21,23,26,30,34,35,41], the more general Conditional FDs (CFDs) [9,13,38], and the more general Denial Constraints (DCs) [4,11,36,37].…”

Section: Introductionmentioning

confidence: 99%

“…Most of the work to date on approximate constraint discovery has focused on approximate FDs [12,23,25] or CFDs [9,13,38]. Chu et al [11] and later Pena et al [36,37] considered approximate DCs.…”

Section: Introductionmentioning

confidence: 99%

Approximate Denial Constraints

Livshits¹,

Heidari²,

Ilyas³

et al. 2020

Preprint

View full text Add to dashboard Cite

The problem of mining integrity constraints from data has been extensively studied over the past two decades for commonly used types of constraints including the classic Functional Dependencies (FDs) and the more general Denial Constraints (DCs). In this paper, we investigate the problem of mining approximate DCs (i.e., DCs that are "almost" satisfied) from data. Considering approximate constraints allows us to discover more accurate constraints in inconsistent databases, detect rules that are generally correct but may have a few exceptions, as well as avoid overfitting and obtain more general and less contrived constraints. We introduce the algorithm ADCMiner for mining approximate DCs. An important feature of this algorithm is that it does not assume any specific definition of an approximate DC, but takes the semantics as input. Since there is more than one way to define an approximate DC and different definitions may produce very different results, we do not focus on one definition, but rather on a general family of approximation functions that satisfies some natural axioms defined in this paper and captures commonly used definitions of approximate constraints. We also show how our algorithm can be combined with sampling to return results with high accuracy while significantly reducing the running time.

show abstract

Revisiting Conditional Functional Dependency Discovery: Splitting the “C” from the “FD”

Cited by 11 publications

References 20 publications

Pattern functional dependencies for data cleaning

Pattern functional dependencies for data cleaning

An Efficient Approach for Discovering Graph Entity Dependencies (GEDs)

Approximate Denial Constraints

Contact Info

Product

Resources

About