Ensemble diversity measures and their application to thinning

Abstract

The diversity of an ensemble of classifiers can be calculated in a variety of ways. Here a diversity metric and a means for altering the diversity of an ensemble, called ''thinning'', are introduced. We evaluate thinning algorithms created by several techniques on 22 publicly available datasets. When compared to other methods, our percentage correct diversity measure shows a greatest correlation between the increase in voted ensemble accuracy and the diversity value. Also, the analysis of different ensemble creation methods indicates that they generate different levels of diversity. Finally, the methods proposed for thinning show that ensembles can be made smaller without loss in accuracy.

Using Diversity for Classifier Ensemble Pruning: An Empirical Investigation

“…In particular, several optimization-based pruning methods use the forward or backward search (FS/BS) strategy [18,19,20,21,22,23,24].…”

“…• A given diversity measure (disregarding the performance of individual classifiers and of the ensemble) [19,20,23].…”

“…• Ad hoc measures specifically devised for ensemble pruning, which combine into a single scalar the individual classifiers' performance and the complementarity (diversity) between their errors [18,22,23,24].…”

“…(5 of 15) It is then interesting to understand whether and under what conditions such evaluation functions are more effective (in terms of the performance of the resulting ensemble) than directly evaluating the performance of the considered ensembles (estimated, e.g., from validation data) during the pruning procedure. Quite surprisingly, so far such a comparison has been carried out by only a few authors [14,19,22,23,24], and only with a limited scope. In particular, it was often limited to the proposed evaluation measure, and using different and incomparable experimental set-up (i.e., different data sets, base classifiers, ensemble construction methods, etc.).…”

Network-constrained forest for regularized classification of omics data

“…Most of the measures evaluating the ensemble diversity before the validation are based on base classifiers disagreement on certain training samples, often called critical samples beyond the ensemble margin [45,46]. When there is only few examples, the diversity might not be observable within training sample.…”