Evolving an Optimal Decision Template for Combining Classifiers

Abstract

In this paper, we aim to develop an effective combining algorithm for ensemble learning systems. The Decision Template method, one of the most popular combining algorithms for ensemble systems, does not perform well when working on certain datasets like those having imbalanced data. Moreover, point estimation by computing the average value on the outputs of base classifiers in the Decision Template method is sometimes not a good representation, especially for skewed datasets. Here we propose to search for an optimal decision template in the combining algorithm for a heterogeneous ensemble. To do this, we first generate the base classifier by training the pre-selected learning algorithms on the given training set. The meta-data of the training set is then generated via cross validation. Using the Artificial Bee Colony algorithm, we search for the optimal template that minimizes the empirical 0-1 loss function on the training set. The class label is assigned to the unlabeled sample based on the maximum of the similarity between the optimal decision template and the sample's meta-data. Experiments conducted on the UCI datasets demonstrated the superiority of the proposed method over several benchmark algorithms.

Weighted Ensemble of Deep Learning Models based on Comprehensive Learning Particle Swarm Optimization for Medical Image Segmentation

“…in which P k (y m |I(i, j)) is the probability that the pixel I(i, j) belongs to the class label y m given by the classifier generated by using K k for each k = 1, ..., K; m = 1, ..., M and [12], [13]. The prediction for all images in the training set D is given by a…”

“…The next step is to train the combining algorithm on P. There are two combining models developed for the ensemble systems, namely representation-based model and weighted combining-based model [13]…”

WEC: Weighted Ensemble of Text Classifiers

“…in which • returns 1 if the condition is true, otherwise returns 0. This loss function is for the classification error rate which is one of the most popular performance metrics in the literature [21]- [23]. The loss on the training set associated with w is given by:…”

A Weighted Ensemble of Regression Methods for Gross Error Identification Problem

“…SELECTION: We utilize the roulette wheel selection procedure to balance fitness-based criteria and randomness [21]. At each generation, a number of parents are selected based on their fitness values to generate offspring.…”