Incremental construction of classifier and discriminant ensembles

Abstract

a b s t r a c tWe discuss approaches to incrementally construct an ensemble. The first constructs an ensemble of classifiers choosing a subset from a larger set, and the second constructs an ensemble of discriminants, where a classifier is used for some classes only. We investigate criteria including accuracy, significant improvement, diversity, correlation, and the role of search direction. For discriminant ensembles, we test subset selection and trees. Fusion is by voting or by a linear model. Using 14 classifiers on 38 data sets, incremental search finds small, accurate ensembles in polynomial time. The discriminant ensemble uses a subset of discriminants and is simpler, interpretable, and accurate. We see that an incremental ensemble has higher accuracy than bagging and random subspace method; and it has a comparable accuracy to AdaBoost, but fewer classifiers.

A Multiple Kernel Learning Algorithm for Cell Nucleus Classification of Renal Cell Carcinoma

“…Here, kernels are combined globally, namely the kernels are assigned the same weights for the whole input space. It has been shown by many researchers that using a subset of given classification algorithms increases accuracy rather than using all the classifiers [12,14]. Keeping this in mind, we apply the same idea to incrementally adding kernels to the MKL framework and compare the results.…”

“…This procedure continues until all kernels (classifiers) are used or the average validation accuracy does not increase [14]. The algorithm starts with E 0 ← ∅, then at each step t, all the kernels (classifiers) M j ∈ E (t−1) are combined with …”

Multimodal Schizophrenia Detection by Multiclassification Analysis

“…One can see that the best accuracy of Con is improved from 71.93 % to 74.56 %. It's known that combining a subset of models may lead to better accuracies than using all the models [10,12]. Bearing this in mind, we have performed another experiment where we selected the best seven ROIs.…”

Dynamic Programming for Bayesian Logistic Regression Learning under Concept Drift

“…Many authors proposed one-pass versions of batch learning algorithms. Later, ensembles of classifiers were additionally applied to the problems which had been originally assumed to be solved by single classifier-based incremental learning algorithms [1], [4,5].…”