A New Ensemble Diversity Measure Applied to Thinning Ensembles

Banfield, Robert E.; Hall, Lawrence O.; Bowyer, Kevin W.; Kegelmeyer, W. Philip

doi:10.1007/3-540-44938-8_31

Cited by 62 publications

(43 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The work presented here extends our previous work [8] by testing additional thinning methods, using more datasets, and evaluating diversity and thinning algorithms on ensembles created by several different algorithms.…”

Section: Introductionmentioning

confidence: 71%

“…We reference the terminology described in [8] to describe the removal of decision trees from a forest. Since this process can be likened to "thinning a forest", we call it "thinning.…”

Section: A Review Of Other Methodsmentioning

confidence: 99%

“…The PCDM algorithm [8] is shown in Figure 2. It works by finding the test set examples for which T low to T high of the individual classifiers in the ensemble are correct.…”

Section: ))mentioning

confidence: 99%

See 2 more Smart Citations

Ensemble diversity measures and their application to thinning

Banfield

2004

Information Fusion

View full text Add to dashboard Cite

Abstract. The diversity of an ensemble can be calculated in a variety of ways. Here a diversity metric and a means for altering the diversity of an ensemble, called "thinning", are introduced. We evaluate thinning algorithms on ensembles created by several techniques on 22 publicly available datasets.When compared to other methods, our percentage correct diversity measure algorithm shows a greater correlation between the increase in voted ensemble accuracy and the diversity value. Also, the analysis of different ensemble creation methods indicates each has varying levels of diversity. Finally, the methods proposed for thinning again show that ensembles can be made smaller without loss in accuracy.

show abstract

Section: Introductionmentioning

confidence: 71%

“…We reference the terminology described in [8] to describe the removal of decision trees from a forest. Since this process can be likened to "thinning a forest", we call it "thinning.…”

Section: A Review Of Other Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Ensemble diversity measures and their application to thinning

Banfield

2004

Information Fusion

View full text Add to dashboard Cite

show abstract

“…Though it does not concern generalisation performance, a surprising result is that the number of trees in the "best" subset found during the selection process is often very small regarding to the size of the initial forest, sometimes even approaching 30% of the amount of available trees (Musk, Segment and Vehicle). Classifier selection has already shown to be a powerful tool for obtaining significant improvement with ensemble of classifiers [25][26][27], but this result leads us to think that it would be interesting to further focus on the number of trees that have to be grown in a forest to obtain significant improvement comparing to RF induced with Forest-RI, and rather according to generalisation accuracy.…”

Section: M(i) ← Run a Mcnemar Test Of Significance With Classifiers (mentioning

confidence: 99%

Towards a Better Understanding of Random Forests through the Study of Strength and Correlation

Bernard

Adam

2009

Emerging Intelligent Computing Technology and Applications. With Aspects of Artificial Intelligence

View full text Add to dashboard Cite

Abstract. In this paper we present a study on the Random Forest (RF) family of ensemble methods. From our point of view, a "classical" RF induction process presents two main drawbacks : (i) the number of trees has to be a priori fixed (ii) trees are independently, thus arbitrarily, added to the ensemble due to the randomization principle. Hence, this kind of process offers no guarantee that all the trees will well cooperate into the same committee. In this work we thus propose to study the RF mechanisms that explain this cooperation by analysing, for particular subsets of trees called sub-forests, the link between accuracy and properties such as Strength and Correlation. We show that these properties, through the Correlation/Strengh 2 ratio, should be taken into account to explain the sub-forest performance.

show abstract

“…Margineantu and Dietterich 30 used kappa statistic to prune the adaptive boosting ensemble to a prespecified size. A similar approach called "thinning the ensemble" can be found in the study by Banfield et al 31 All these techniques start with a set of subclassifiers and select a subset to construct the final ensemble. Note that the size of the initial ensemble needs to be specified by the designer.…”

Section: Introductionmentioning

confidence: 99%

An adaptive incremental approach to constructing ensemble classifiers: Application in an information‐theoretic computer‐aided decision system for detection of masses in mammograms

2009

View full text Add to dashboard Cite

Ensemble classifiers have been shown efficient in multiple applications. In this article, the authors explore the effectiveness of ensemble classifiers in a case-based computer-aided diagnosis system for detection of masses in mammograms. They evaluate two general ways of constructing subclassifiers by resampling of the available development dataset: Random division and random selection. Furthermore, they discuss the problem of selecting the ensemble size and propose two adaptive incremental techniques that automatically select the size for the problem at hand. All the techniques are evaluated with respect to a previously proposed information-theoretic CAD system ͑IT-CAD͒. The experimental results show that the examined ensemble techniques provide a statistically significant improvement ͑AUC= 0.905Ϯ 0.024͒ in performance as compared to the original IT-CAD system ͑AUC= 0.865Ϯ 0.029͒. Some of the techniques allow for a notable reduction in the total number of examples stored in the case base ͑to 1.3% of the original size͒, which, in turn, results in lower storage requirements and a shorter response time of the system. Among the methods examined in this article, the two proposed adaptive techniques are by far the most effective for this purpose. Furthermore, the authors provide some discussion and guidance for choosing the ensemble parameters.

show abstract

A New Ensemble Diversity Measure Applied to Thinning Ensembles

Cited by 62 publications

References 9 publications

Ensemble diversity measures and their application to thinning

Ensemble diversity measures and their application to thinning

Towards a Better Understanding of Random Forests through the Study of Strength and Correlation

An adaptive incremental approach to constructing ensemble classifiers: Application in an information‐theoretic computer‐aided decision system for detection of masses in mammograms

Contact Info

Product

Resources

About