Hybrid semiparametric Bayesian networks

Abstract

This paper presents a new class of Bayesian networks called hybrid semiparametric Bayesian networks, which can model hybrid data (discrete and continuous data) by mixing parametric and nonparametric estimation models. The parametric estimation models can represent a conditional linear Gaussian relationship between variables, while the nonparametric estimation model can represent other types of relationships, such as non-Gaussian and nonlinear relationships. This new class of Bayesian networks generalizes the conditional linear Gaussian Bayesian networks, including them as a special case. In addition, we describe a learning procedure for the structure and the parameters of our proposed type of Bayesian network. This learning procedure finds the best combination of parametric and nonparametric models automatically from data. This requires the definition of a cross-validated score. We also detail how new data can be sampled from a hybrid semiparametric Bayesian network, which in turn can be useful to solve other related tasks, such as inference. Furthermore, we intuitively relate our proposal with adaptive kernel density estimation models. The experimental results show that hybrid semiparametric Bayesian networks are a valuable contribution when dealing with data that do not meet the parametric assumptions that are expected for other models, such as conditional linear Gaussian Bayesian networks. We include experiments with synthetic data and real-world data from the UCI repository which demonstrate the good performance and the ability to extract useful information about the relationship between the variables in the model.

Estimation of Distribution Algorithms in Machine Learning: A Survey

“…EDAs based on Bayesian networks might adopt modern learning algorithms [204] providing better data fitting structures (such as semiparametric Bayesian networks [205], [206], in which nodes estimated by kernels coexist with nodes that assume Gaussianity) and better efficiency could result in the improved performance of EDAs. This has been done in the new SPEDA [207], based on semiparametric Bayesian networks, which turned out to be one of the best performing approaches with respect to other state-of-the-art algorithms in continuous optimization.…”

Modular Graph Transformer Networks for Multi-Label Image Classification

“…Throughout, we assume the chain rule of probability [39][Section 2.1.3.4] holds 3 . Using the concept of conditional independence, we can assume without loss of generality that any p(X, Y) can be fully encoded by a DAG G and a parameter set θ inducing the factorization…”

“…A key point is the separation of discrete conditionals π and continuous conditionals x c . Such separations were used in learning BNs optimizing the likelihood function [3] Moreover, we have…”

“…This formal framework allows us to learn an optimal multi-dimensional classifier, without loss of generality/optimality, by decomposing the task into learning a set of probabilistic MCC models plus a directed acyclic graph (DAG). Notably, the framework inherits the interpretability of Bayesian networks (BNs) [3,37,39], which is a compact representation of quantitative and qualitative probabilistic relationships among class variables, and the scalability and flexibility of deep (multimodal) learning [38,41,69], i.e., handling complex types of data. Moreover, the probabilistic nature allows the framework, among other characteristics, to optimize different loss functions by only learning a single probabilistic model.…”