Data clustering using hidden variables in hybrid Bayesian networks

Abstract

In this paper, we analyze the problem of data clustering in domains where discrete and continuous variables coexist. We propose the use of hybrid Bayesian networks with naïve Bayes structure and hidden class variable. The model integrates discrete and continuous features, by representing the conditional distributions as mixtures of truncated exponentials (MTEs). The number of classes is determined through an iterative procedure based on a variation of the data augmentation algorithm. The new model is compared with an EM-based clustering algorithm where each class model is a product of conditionally independent probability distributions and the number of clusters is decided by using a cross-validation scheme. Experiments carried out over real-world and synthetic data sets show that the proposal is competitive with state-of-the-art methods. Even though the methodology introduced in this manuscript is based on the use of MTEs, it can be easily instantiated to other similar models, like the Mixtures of Polynomials or the mixtures of truncated basis functions in general.

Mixture-Based Probabilistic Graphical Models for the Label Ranking Problem

“…This idea is not new, and has been used, for instance, for unsupervised clustering [ 21 , 31 ], to improve the performance (accuracy) of the base classifier [ 32 ], relax some of the independence statements increasing the classifier modeling capability [ 33 , 34 , 35 ], or obtain models for efficient probabilistic inference [ 36 ].…”

“…The proposed probabilistic LR-classifier relies on the use of a hybrid Bayesian network [ 21 ] where different probability distributions are used to conveniently model variables of a different nature: Multinomial for discrete variables, Gaussian for numerical variables, and Mallows for permutations [ 22 ]. The Mallows probability distribution is usually considered to model a set of permutations and, in fact, is the core of the decision tree algorithm (LRT) proposed in [ 2 ].…”

A Soft Clustering Approach to Detect Socio-Ecological Landscape Boundaries Using Bayesian Networks

“…In this regard, the goal of an unsupervised classifier is to find groups of elements based on their similarities. In this work, we follow the methodology proposed in [31,38] (Algorithm 1), which details the specific steps and algorithms, implemented in Elvira software [39]. In this approach, the class variable C is replaced by a hidden variable, H, whose values are initially missing.…”

Analysis of the socioecological structure and dynamics of the territory using a hybrid Bayesian network classifier

“…2ii). They are based on the probabilistic clustering methodology using HBNs as proposed by Fernández et al (2014), and implemented in the Elvira software (Elvira-Consortium, 2002). Fig.…”

“…A classification problem in which no information about the class variable is available (called an unsupervised classification or clustering problem) can be solved by a BN classifier (Aguilera et al, 2013;Anderberg, 1973;Fernández et al, 2014;Gieder et al, 2014). This soft-clustering methodology implies the partition of the data into groups in such a way that the observations belonging to one group are similar to each other but differ from the observations in the other groups.…”