Supervised classification with conditional Gaussian networks: Increasing the structure complexity from naive Bayes

Abstract

Most of the Bayesian network-based classifiers are usually only able to handle discrete variables. However, most real-world domains involve continuous variables. A common practice to deal with continuous variables is to discretize them, with a subsequent loss of information. This work shows how discrete classifier induction algorithms can be adapted to the conditional Gaussian network paradigm to deal with continuous variables without discretizing them. In addition, three novel classifier induction algorithms and two new propositions about mutual information are introduced. The classifier induction algorithms presented are ordered and grouped according to their structural complexity: naive Bayes, tree augmented naive Bayes, k-dependence Bayesian classifiers and semi naive Bayes. All the classifier induction algorithms are empirically evaluated using predictive accuracy, and they are compared to linear discriminant analysis, as a continuous classic statistical benchmark classifier. Besides, the accuracies for a set of state-of-the-art classifiers are included in order to justify the use of linear discriminant analysis as the benchmark algorithm. In order to understand the behavior of the conditional Gaussian network-based classifiers better, the results include bias-variance decomposition of the expected misclassification rate. The study suggests that semi naive Bayes structure based classifiers and, especially, the novel wrapper condensed semi naive Bayes backward, outperform the behavior of the rest of the presented classifiers. They also obtain quite competitive results compared to the state-of-the-art algorithms included.

Directional naive Bayes classifiers

“…This is a frequent situation when we can measure both the magnitude and the direction of a given phenomenon, e.g., the direction and the velocity of wind currents or the strength and orientation of a magnetic field. We study the hybrid NB classifier where the directional variable X is modeled using von Mises-Fisher distributions and the linear variable Y is modeled using The multivariate Gaussian distribution Af(fi, £) is defined by its two parameters: the mean fi and the covariance matrix E. The decision function r(Y) of a Gaussian NB [59] found by substituting this probability density function in (3) is:…”

“…SelNB finds the variables inducing the most accurate NB structure in a wrapper fashion. Perez et al [59] proposed a filterwrapper approach to induce SelNB classifiers. First, the filter algorithm ranks the predictive variables using the mutual information (MI) between each variable and the class.…”

Fuzzy Naive Bayesian for constructing regulated network with weights

“…All other attributes nodes are dependent on the class attribute nodes, in addition, other attributes are mutual independence [3]. Because of its highly conditional independence assumption, the Naive Bayesian classifier makes entire classifier training, and makes the classification process simplified, and its structure fixed.…”

Monitoring of Complex Processes with Bayesian Networks

“…An other interesting approach is the Kononenko one (Kononenko (1991)), which represents some variables in one node. As in (Perez et al (2006)) the assumption we will make is that this variable follows a normal multivariate distribution (conditionally to the class) and we will refer to this kind of BN as Condensed Semi Naïve Bayesian Network (CSNBN). …”