Clustering-based Bayesian Multi-net Classifier construction with Ant Colony Optimization

Abstract

Abstract-Bayesian Multi-nets (BMNs) are a special kind of Bayesian network (BN) classifiers that consist of several local networks, typically, one for each predictable class, to model an asymmetric set of variable dependencies given each class value. Alternatively, multi-nets can be learnt upon arbitrary partitions of a dataset, in which each partition holds more consistent variable dependencies given the data subset in the partition. This paper proposes two contributions to the approach that clusters the dataset into separate data subsets to build asymmetric local BN classifiers, one for each subset. First, we extend the K-modes algorithm, previously used by the Case-Based Bayesian Network Classifiers (CBBN) approach to create clusters before learning the BN classifiers. Second, we introduce the Ant-Clust-B algorithm that employs Ant Colony Optimization (ACO) to learn clusteringbased BMNs. Ant-Clust-B uses ACO in the clustering step before learning the local BN classifiers. Empirical results are obtained from experiments on 18 UCI datasets. [12]. Recently, the authors have introduced ABC-Miner [13], the first ACO-based algorithm to build Bayesian network classifiers, which has shown better performance compared to some greedy and deterministic BN algorithms. Thus, we carry on developing ACO-based algorithms in the Bayesian classification area. Classification is a central problem in data mining and machine learning where the system builds, from labelled data instances, a model (classifier) that predicts the class of unlabelled instances [14]. There are many types of classification methods [14], but in this work we focus on building BN classifiers. I. INTRODUCTIONA BN classifier is a special kind of probabilistic networks that aims to predict the class of a data instance by computing the posterior probability of each available class value, given the values of the predictor attributes of the instance, and then labeling the instance with the class having the highest posterior probability. Naïve-Bayes, as discussed in [15], is the simplest kind of BN classifier and it assumes the attributes are independent given the class label. Although it obtained good predictive performance in several domains [15], extensions

Ant colony algorithms for constructing Bayesian multi-net classifiers

“…Cluster-based BMN refers to partitioning the dataset into arbitrary data subsets, and learning a local BN classifier for each subset. While the procedure of ABC-Miner is comparable to the algorithms proposed in our current work, the ACO clustering-based BMN algorithm [41] is very different for the following reason. In the current proposed algorithms, ACO is employed for finding the optimal structure of the local BNs -similar to ABC-Miner, in which the ACO algorithm optimizes the structure of BAN classifiers.…”

“…Besides, we have recently introduced an ACO-based algorithm that learns cluster-based BMNs [41], rather than learning class-based BMNs; the focus of this current work. Cluster-based BMN refers to partitioning the dataset into arbitrary data subsets, and learning a local BN classifier for each subset.…”

“…In the current proposed algorithms, ACO is employed for finding the optimal structure of the local BNs -similar to ABC-Miner, in which the ACO algorithm optimizes the structure of BAN classifiers. On the other hand, in [41], ACO is utilized to produce the data clusters, rather than learning the local BN classifiers, while a simple Naïve-Bayes classifier is constructed for each data cluster.…”