Classification using Hierarchical Naïve Bayes models

Abstract

Classification problems have a long history in the machine learning literature. One of the simplest, and yet most consistently well-performing set of classifiers is the Naïve Bayes models. However, an inherent problem with these classifiers is the assumption that all attributes used to describe an instance are conditionally independent given the class of that instance. When this assumption is violated (which is often the case in practice) it can reduce classification accuracy due to "information double-counting" and interaction omission.In this paper we focus on a relatively new set of models, termed Hierarchical Naïve Bayes models. Hierarchical Naïve Bayes models extend the modeling flexibility of Naïve Bayes models by introducing latent variables to relax some of the independence statements in these models. We propose a simple algorithm for learning Hierarchical Naïve Bayes models in the context of classification. Experimental results show that the learned models can significantly improve classification accuracy as compared to other frameworks.

Integrating non-animal test information into an adaptive testing strategy – skin sensitization proof of concept case

“…the BN ItS structure represents a Hierarchical Naïve Bayes classifier (Langseth and Nielsen, 2006) except that the tIMeS node is connected to both Reactivity cluster and directly to the hypothesis variable llNA (Fig. 3).…”

Predicting customer responses to direct marketing : a Bayesian approach

“…Bayes (Langseth and Nielsen, 2006). In our study, we do not use the Hierarchical Bayes to generate the latent variables which is currently impractical.…”

Latent tree models : an application and an extension

“…To improve the classification accuracy of the resulting models, Langseth and Nielsen [95] propose to use the classification accuracy for model selection during the search. The classification accuracy is estimated by cross-validation, so the testing data need not be used.…”