Discovering Graph Functional Dependencies

Fan, Wenfei; Hu, Chunming; Liu, Xueli; Lü, Ping

doi:10.1145/3183713.3196916

Cited by 25 publications

(4 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Data profiling is also heavily used in graph data, e.g., using Petri Nets in process mining to recover missing events [105], [106] and clean event data [107], discovering keys for graphs and applying keys to study entity matching [108], or defining functional dependencies for graphs [21] and discovering them [109]. However, the above studies still seem to be difficult when encountering large graphs.…”

Section: B Sampling For Profiling Graph Datamentioning

confidence: 99%

“…In this case, one has to design two parallel scalable algorithms, in MapReduce and a vertex-centric asynchronous model. In order to find Graph Functional Dependencies, Fan et al [109] have to deal with large-scale graphs by designing effective pruning strategies, using parallel algorithms, and adding processors. As mentioned earlier, designing parallel algorithms is difficult.…”

Section: B Sampling For Profiling Graph Datamentioning

confidence: 99%

See 1 more Smart Citation

Sampling for Big Data Profiling: A Survey

Liu

Zhang

2020

IEEE Access

View full text Add to dashboard Cite

Due to the development of internet technology and computer science, data is exploding at an exponential rate. Big data brings us new opportunities and challenges. On the one hand, we can analyze and mine big data to discover hidden information and get more potential value. On the other hand, the 5V characteristic of big data, especially Volume which means large amount of data, brings challenges to storage and processing. For some traditional data mining algorithms, machine learning algorithms and data profiling tasks, it is very difficult to handle such a large amount of data. The large amount of data is highly demanding hardware resources and time consuming. Sampling methods can effectively reduce the amount of data and help speed up data processing. Sampling technology has been widely used in big data context. Data profiling is the activity that finds metadata of data set and has many use cases, e.g., performing data profiling tasks on relational data, graph data, and time series data for anomaly detection and data repair. However, data profiling is computationally expensive, especially for large data sets. Hence this article focuses on researching sampling for data profiling tasks in big data context and investigates the application of sampling in different categories of data profiling. From the experimental results of these studies, the results got from the sampled data are close to or even exceed the results of the full amount of data. Therefore, sampling technology plays an important role in the era of big data, and we also have reason to believe that sampling technology will become an indispensable step in big data processing in the future.INDEX TERMS Big data, large amount, sampling, data profiling.

show abstract

Section: B Sampling For Profiling Graph Datamentioning

confidence: 99%

Section: B Sampling For Profiling Graph Datamentioning

confidence: 99%

Sampling for Big Data Profiling: A Survey

Liu

Zhang

2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…This differs from our approach, which is specifically developed for graphs. More similar methods are used by (Fan et al, 2018;Yu and Heflin, 2011), which start out with minimal constraints and extend these iteratively until all options are exhausted. However, these algorithms only consider FDs.…”

Section: Related Workmentioning

confidence: 99%

Bottom-up Discovery of Context-aware Quality Constraints for Heterogeneous Knowledge Graphs

Wilcke

Kleijn

Boer

et al. 2020

Proceedings of the 12th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management

View full text Add to dashboard Cite

General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal Take down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.

show abstract

“…NGDs. We extended the algorithm of [22] to discover NGDs from the graphs. The algorithm interleaves "vertical levelwise expansion" for mining frequent patterns Q and "horizontal levelwise expansion" for mining literals in X → Y , in a single process.…”

Section: Experimental Studymentioning

confidence: 99%

Catching Numeric Inconsistencies in Graphs

Fan

Liu

Lü

et al. 2018

Proceedings of the 2018 International Conference on Management of Data

Self Cite

View full text Add to dashboard Cite

Numeric inconsistencies are common in real-life knowledge bases and social networks. To catch such errors, we propose to extend graph functional dependencies with linear arithmetic expressions and comparison predicates, referred to as NGDs. We study fundamental problems for NGDs. We show that their satisfiability, implication and validation problems are Σ p 2 -complete, Π p 2 -complete and coNP-complete, respectively. However, if we allow non-linear arithmetic expressions, even of degree at most 2, the satisfiability and implication problems become undecidable. In other words, NGDs strike a balance between expressivity and complexity.To make practical use of NGDs, we develop an incremental algorithm IncDect to detect errors in a graph G using NGDs, in response to updates ∆G to G. We show that the incremental validation problem is coNP-complete. Nonetheless, algorithm IncDect is localizable, i.e., its cost is determined by small neighbors of nodes in ∆G instead of the entire G. Moreover, we parallelize IncDect such that it guarantees to reduce running time with the increase of processors. Using real-life and synthetic graphs, we experimentally verify the scalability and efficiency of the algorithms.

show abstract

Discovering Graph Functional Dependencies

Cited by 25 publications

References 45 publications

Sampling for Big Data Profiling: A Survey

Sampling for Big Data Profiling: A Survey

Bottom-up Discovery of Context-aware Quality Constraints for Heterogeneous Knowledge Graphs

Catching Numeric Inconsistencies in Graphs

Contact Info

Product

Resources

About