Incremental Learning of Latent Forests

Abstract

In the analysis of real-world data, it is useful to learn a latent variable model that represents the data generation process. In this setting, latent tree models are useful because they are able to capture complex relationships while being easily interpretable. In this paper, we propose two incremental algorithms for learning forests of latent trees. Unlike current methods, the proposed algorithms are based on the variational Bayesian framework, which allows them to introduce uncertainty into the learning process and work with mixed data. The first algorithm, incremental learner, determines the forest structure and the cardinality of its latent variables in an iterative search process. The second algorithm, constrained incremental learner, modifies the previous method by considering only a subset of the most prominent structures in each step of the search. Although restricting each iteration to a fixed number of candidate models limits the search space, we demonstrate that the second algorithm returns almost identical results for a small fraction of the computational cost. We compare our algorithms with existing methods by conducting a comparative study using both discrete and continuous real-world data. In addition, we demonstrate the effectiveness of the proposed algorithms by applying them to data from

Multipartition clustering of mixed data with Bayesian networks

“…We ran 1000 iterations of the sampler using the Python implementation available at https://github.com/ ivaleraM/GLFM. • Incremental learner (IL), 50 which hill-climbs the space of latent forests in a two-phase iterative process. In its first phase, the forest structure is incremented with a new arc or latent variable.…”

“…In the local VB-SEM algorithm, we allow adding, removing, or reversing arcs that contain any of the variables considered by the latent operator. Once the structure has been learned, the local VB-EM algorithm 50 estimates the parameters of those variables belonging to the Markov blankets (MBs) of: (i) variables initially selected by the latent operator, and (ii) variables affected by a structure change (e.g., a new incoming arc). For any variable in a BN, its MB consists of the set of all its parents, children, and spouses (parents of children) in the network.…”

Discovery and density estimation of latent confounders in Bayesian networks with evidence lower bound

“…VBEM iterates over the VB-E and VB-M steps until the difference in ELBO becomes smaller than a given threshold, indicating convergence. A revised version called p-ELBO was proposed by Rodriguez-Sanchez et al (2020) that includes a penalty term to avoid the |L i |! equivalent ways of assigning sets of parameters that result in the same distribution (non-identifiability), and it is defined as p…”

“…The ELBO score was extended to p-ELBO by Rodriguez-Sanchez et al (2020;2022), which was used as the objective score in Constrained Incremental Learner (CIL) and Greedy Latent Structure Learner (GLSL) algorithms. CIL learns a tree-structured BN that assumes any two nodes are connected by one directed path only, whereas GLSL learns a DAG BN.…”