Approximating probabilistic inference in Bayesian belief networks is NP-hard

Dagum, Paul; Luby, Michael

doi:10.1016/0004-3702(93)90036-b

Cited by 553 publications

(260 citation statements)

References 11 publications

Supporting

Mentioning

242

Contrasting

Unclassified

Order By: Relevance

“…A variety of Monte Carlo algorithms have been developed (see Neal, 1993) and applied to the inference problem in graphical models (Dagum & Luby, 1993;Fung & Favero, 1994;Gilks, Thomas, & Spiegelhalter, 1994;Jensen, Kong, & Kjaerulff, 1995;Pearl, 1988). Advantages of these algorithms include their simplicity of implementation and theoretical guarantees of convergence.…”

Section: P(h | E) = P(h E) P(e)mentioning

confidence: 99%

Untitled

Jordan

Ghahramani

Jaakkola³

et al. 1999

Machine Learning

2,111

175

View full text Add to dashboard Cite

Abstract. This paper presents a tutorial introduction to the use of variational methods for inference and learning in graphical models (Bayesian networks and Markov random fields). We present a number of examples of graphical models, including the QMR-DT database, the sigmoid belief network, the Boltzmann machine, and several variants of hidden Markov models, in which it is infeasible to run exact inference algorithms. We then introduce variational methods, which exploit laws of large numbers to transform the original graphical model into a simplified graphical model in which inference is efficient. Inference in the simpified model provides bounds on probabilities of interest in the original model. We describe a general framework for generating variational transformations based on convex duality. Finally we return to the examples and demonstrate how variational algorithms can be formulated in each case.

show abstract

Section: P(h | E) = P(h E) P(e)mentioning

confidence: 99%

Untitled

Jordan

Ghahramani

Jaakkola³

et al. 1999

Machine Learning

2,111

175

View full text Add to dashboard Cite

show abstract

“…A variety o f M o n te Carlo algorithms have been developed see MacKay, this volume, and Neal, 1993 and applied to the inference problem in graphical models Dagum & Luby, 1993;Fung & Favero, 1994;Gilks, Thomas, & Spiegelhalter, 1994;Jensen, Kong, & Kj rul , 1995;Pearl, 1988 . Advantages of these algorithms include their simplicity of implementation and theoretical guarantees of convergence.…”

Section: Introductionmentioning

confidence: 99%

An Introduction to Variational Methods for Graphical Models

Jordan

Ghahramani

Jaakkola

et al. 1998

Learning in Graphical Models

1,585

2,090

View full text Add to dashboard Cite

Abstract. This paper presents a tutorial introduction to the use of variational methods for inference and learning in graphical models. We present a number of examples of graphical models, including the QMR-DT database, the sigmoid belief network, the Boltzmann machine, and several variants of hidden Markov models, in which it is infeasible to run exact inference algorithms. We then introduce variational methods, showing how upper and lower bounds can be found for local probabilities, and discussing methods for extending these bounds to bounds on global probabilities of interest. Finally we return to the examples and demonstrate how variational algorithms can be formulated in each case.

show abstract

“…Approximate methods for inference in Bayesian networks with other distributions, such as the generalized linear-regression model, have also been developed Saul et al, 1996;Jaakkola and Jordan, 1996 . Although we use conditional independence to simplify probabilistic inference, exact inference in an arbitrary Bayesian network for discrete variables is NP-hard Cooper, 1990 . Even approximate inference for example, Monte-Carlo methods is NP-hard Dagum and Luby, 1993 . The source of the di culty lies in undirected cycles in the Bayesian-network structure|cycles in the structure where we ignore the directionality of the arcs.…”

Section: Inference In a Bayesian Networkmentioning

confidence: 99%

A Tutorial on Learning with Bayesian Networks

Heckerman

Studies in Computational Intelligence

980

961

View full text Add to dashboard Cite

A companion set of lecture slides is available at ftp: ftp.research.microsoft.com pub dtg david tutorial.ps. AbstractA B a yesian network is a graphical model that encodes probabilistic relationships among variables of interest. When used in conjunction with statistical techniques, the graphical model has several advantages for data analysis. One, because the model encodes dependencies among all variables, it readily handles situations where some data entries are missing. Two, a Bayesian network can be used to learn causal relationships, and hence can be used to gain understanding about a problem domain and to predict the consequences of intervention. Three, because the model has both a causal and probabilistic semantics, it is an ideal representation for combining prior knowledge which often comes in causal form and data. Four, Bayesian statistical methods in conjunction with Bayesian networks o er an e cient and principled approach for avoiding the over tting of data. In this paper, we discuss methods for constructing Bayesian networks from prior knowledge and summarize Bayesian statistical methods for using data to improve these models. With regard to the latter task, we describe methods for learning both the parameters and structure of a Bayesian network, including techniques for learning with incomplete data. In addition, we relate Bayesian-network methods for learning to techniques for supervised and unsupervised learning. We illustrate the graphical-modeling approach using a real-world case study.

show abstract

Approximating probabilistic inference in Bayesian belief networks is NP-hard

Cited by 553 publications

References 11 publications

Untitled

Untitled

An Introduction to Variational Methods for Graphical Models

A Tutorial on Learning with Bayesian Networks

Contact Info

Product

Resources

About