Bayesian networks for supporting query processing over incomplete autonomous databases

Abstract

As the information available to lay users through autonomous data sources continues to increase, mediators become important to ensure that the wealth of information available is tapped effectively. A key challenge that these information mediators need to handle is the varying levels of incompleteness in the underlying databases in terms of missing attribute values. Existing approaches such as QPIAD aim to mine and use Approximate Functional Dependencies (AFDs) to predict and retrieve relevant incomplete tuples. These approaches make independence assumptions about missing values-which critically hobbles their performance when there are tuples containing missing values for multiple correlated attributes. In this paper, we present a principled probabilistic alternative that views an incomplete tuple as defining a distribution over the complete tuples that it stands for. We learn this distribution in terms of Bayes networks. Our approach involves mining/"learning" Bayes networks from a sample of the database, and using it to do both imputation (predict a missing value) and query rewriting (retrieve relevant results with incompleteness on the query-constrained attributes, when the data sources are autonomous). We present empirical studies to demonstrate that (i) at higher levels of incompleteness, when multiple attribute values are missing, Bayes networks do provide a significantly higher classification accuracy and (ii) the relevant possible answers retrieved by the queries reformulated using Bayes networks provide higher precision and recall than AFDs while keeping query processing costs manageable.

A Bayesian approach to estimate and model SINR in wireless networks

“…There are several different engines for performing inference in Bayesian networks that make different tradeoffs between speed, complexity, generality, and accuracy. In this article, we use the junction‐tree engine …”

“…In this article, we use the junction-tree engine. 23 In our model shown in Figure 2, there are no direct relationships between SINR and other variables such as distance, frequency, path loss, shadow fading, and transmission powers. Therefore, an inference engine is required to infer the CPD of SINR based on the CPD and relationships of the other variables in the model.…”

Dissociation and propagation for approximate lifted inference with standard relational database management systems

“…Recent large knowledge bases, such as Yago [40], Nell [7], DeepDive [16], or Google's Knowledge Vault [19], have millions to billions of uncertain tuples. Data sets with missing values are often "completed" using inference in graphical models [8,61,71] or sophisticated low rank matrix factorization techniques [21,70] that ultimately result in a large probabilistic database. Data sets that result from crowdsourcing [1] or that are inferred from unstructured information [9] are also uncertain, and probabilistic databases have been applied to bootstrapping over samples of data [80].…”