Structure learning of Bayesian networks using constraints

Abstract

This paper addresses exact learning of Bayesian network structure from data and expert's knowledge based on score functions that are decomposable. First, it describes useful properties that strongly reduce the time and memory costs of many known methods such as hill-climbing, dynamic programming and sampling variable orderings. Secondly, a branch and bound algorithm is presented that integrates parameter and structural constraints with data in a way to guarantee global optimality with respect to the score function. It is an any-time procedure because, if stopped, it provides the best current solution and an estimation about how far it is from the global solution. We show empirically the advantages of the properties and the constraints, and the applicability of the algorithm to large data sets (up to one hundred variables) that cannot be handled by other current methods (limited to around 30 variables).

Widened Learning of Bayesian Network Classifiers

“…Significantly, the use of the Frobenius Norm of the difference of the Bayesian networks's graph Laplacians is very encouraging and suggests further research into distance measures based on graph features such as those derived from Spectral Graph Theory. Experiments with alternate starting states based on conditional information, in a manner similar to the PC Algorithm and CBL, or constraint-based algorithms like Incremental Association or HITON, or even to those claiming to find the exact network structure [10] could also be promising.…”

Extended Tree Augmented Naive Classifier

“…The optimality of such an idea can be easily proven by the following lemma, which guarantees that we should use only X 0 as parent of X i every time such choice is better than using {X 0 , X j }. It is a straightforward generalization of Lemma 1 in [16]. Lemma 1.…”

Bayesian network data imputation with application to survival tree analysis

“…Past literature experiments (Cussens, 2011;de Campos and Ji, 2011;Jaakkola et al, 2010) indicate that DP is the fastest method for small values of m (fewer than 15 − 20 variables), IP is the best method from 15 − 20 to a hundred, and IP and BB are anytime algorithms, so can be run even with large datasets, and then the accuracy keeps improving with time. K2 is the only non-exact method (that is, not guaranteed to converge to a global maximum solution), but it is very efficient, so we try to use the K2 search as much as possible, that is, we use K2 as long as it can find an improving solution with respect to the previous run (if it cannot, then a globally optimal method is used).…”

“…These methods are mostly based on turning the incomplete data into a complete dataset (or even directly updating the sufficient statistics), and then recurring to particular methods for complete data. We adopt a meta-search composed of a few distinct methods (Jaakkola et al, 2010;de Campos and Ji, 2011;Cooper and Herskovits, 1992;Silander and Myllymaki, 2006) that selects the best procedure to run depending on the number of covariates and running-time of the methods. The idea is to improve the score in the most efficient way, still with the guarantee of achieving optimality.…”