Ancestor Relations in the Presence of Unobserved Variables

Abstract

Abstract. Bayesian networks (BNs) are an appealing model for causal and noncausal dependencies among a set of variables. Learning BNs from observational data is challenging due to the nonidentifiability of the network structure and model misspecification in the presence of unobserved (latent) variables. Here, we investigate the prospects of Bayesian learning of ancestor relations, including arcs, in the presence and absence of unobserved variables. An exact dynamic programming algorithm to compute the respective posterior probabilities is developed, under the complete data assumption. Our experimental results show that ancestor relations between observed variables, arcs in particular, can be learned with good power even when a majority of the involved variables are unobserved. For comparison, deduction of ancestor relations from single maximum a posteriori network structures or their Markov equivalence class appears somewhat inferior to Bayesian averaging. We also discuss some shortcomings of applying existing conditional independence test based methods for learning ancestor relations.

Discovery of Causal Models that Contain Latent Variables Through Bayesian Scoring of Independence Constraints

“…However, these methods do not search over the space of all CBNs that include a given set of measured variables. Rather, they require that the user manually provides the proposed CBN models to be scored [19], they search a very restricted space of models, such as bipartite graphs [23] or trees of hidden structure [7, 16], or they score ancestral relations between pairs of variables [28]. Thus, within a Bayesian framework, the automated discovery of CBNs that contain hidden variables remains an important open problem.…”

Structure learning in Bayesian networks and session analysis of people search within a professional social network

“…The third approach of learning BN structures is the Bayesian model averaging approach (Madigan and York, 1995;Friedman and Koller, 2003;Koivisto and Sood, 2004;Koivisto, 2006;Eaton and Murphy, 2007;Grzegorczyk and Husmeier, 2008;Parviainen and Koivisto, 2011;Niinimaki et al, 2011). Instead of learning and then using only one single DAG (i.e., BN structure), this approach intends to make the prediction based on a set of all the possible DAGs.…”

“…Several of these state-of-the-art algorithms work in the order space, including the exact algorithms (Koivisto and Sood, 2004;Koivisto, 2006;Parviainen and Koivisto, 2011) and the approximate algorithms: the Order MCMC (Friedman and Koller, 2003) and the Partial Order MCMC (Niinimaki et al, 2011). They all assume a special form of the structure prior, termed as order-modular prior (Friedman and Koller, 2003;Koivisto and Sood, 2004), for computational convenience.…”

“…Whether a computed posterior with the bias from the order-modular prior is inferior to its counterpart without such a bias depends on the application scenario and is beyond the scope of this paper. For the detailed discussion about this issue, please see the related papers (Friedman and Koller, 2003;Grzegorczyk and Husmeier, 2008;Parviainen and Koivisto, 2011). One method that helps the Order MCMC (Friedman and Koller, 2003) to correct this bias was proposed by Ellis and Wong (2008).…”

“…Moreover, the estimator based on our DDS algorithm has several desirable properties; for example, unlike these MCMC algorithms, the quality of our estimator can be guaranteed by controlling the number of DAGs sampled by our DDS algorithm. The main contribution of our DDS algorithm is to solve the limitation of the exact DP algorithms (Koivisto and Sood, 2004;Koivisto, 2006;Parviainen and Koivisto, 2011) (whose usage is restricted to modular features or path features for moderate n) to estimate the posteriors of various non-modular features arbitrarily specified by users. Additionally our DDS algorithm can also be used to efficiently perform data prediction tasks in estimating p(x|D) for a large number of data cases.…”