Quantifying the uncertainty of a belief net response: Bayesian error-bars for belief net inference

Abstract

A Bayesian belief network models a joint distribution over variables using a DAG to represent variable dependencies and network parameters to represent the conditional probability of each variable given an assignment to its immediate parents. Existing algorithms assume each network parameter is fixed. From a Bayesian perspective, however, these network parameters can be random variables that reflect uncertainty in parameter estimates, arising because the parameters are learned from data, or as they are elicited from uncertain experts. Belief networks are commonly used to compute responses to queriesi.e., return a number for P(H = h | E = e). Parameter uncertainty induces uncertainty in query responses, which are thus themselves random variables. This paper investigates this query response distribution, and shows how to accurately model it for any query and any network structure. In particular, we prove that the query response is asymptotically Gaussian and provide its mean value and asymptotic variance. Moreover, we present an algorithm for computing these quantities that has the same worst-case complexity as inference in general, and also describe straight-line code when the query includes all n variables. We provide empirical evidence that (1) our approximation of the variance is very accurate, and (2) a Beta distribution with these moments provides a very accurate model of the observed query response distribution. We also show how to use this to produce accurate error bars around these responsesi.e., to determine that the response to P(H = h | E = e) is x ± y with confidence 1 − δ.

Habitat suitability modelling of rare species using Bayesian networks: Model evaluation under limited data

“…This can help verify whether or not the model conforms to expectations. If adequate data were available, 'Bayesian error bars' can be estimated as approximate credible intervals around the conditional probabilities, and subsequently, the model outputs (Van Allen et al, 2008). It is worth noting that confidence in the probabilities is difficult to measure under limited data, but can be obtained as part of expert elicitation.…”

Evaluating the Uncertainty of a Bayesian Network Query Response by Using Joint Probability Distribution

“…Allen et al [33] proposed a method to calculate the amount in equation (2) by using intermediate results that generated by using bucket elimination inference method to calculate the query response.The essential of the conventional method can be explained as follow.…”

“…Here, H is the queried variable, E is the set of evidence variables, h and e are assignments of H and E. The posterior probability P (H = h|E = e) is called query response in this research. Equation (1) proposed by Allen et al [33] is used to evaluate the variance of the query response in this research.…”

Methods for Constructing Uncertainty Intervals for Queries of Bayesian Nets

“…Among those who have are Van Allen, Greiner & Hooper (2001) and Van Allen, Singh, Greiner & Hooper (2008) who develop algorithms for the estimation of the variance of a BN query and use the normal approximation to construct a confidence interval (CI). Hooper et al (2009) develop additional methods for finding the variance of a BN response.…”