A method for dynamic ensemble selection based on a filter and an adaptive distance to improve the quality of the regions of competence

Cruz, Rafael M. O.; Cavalcanti, George D. C.; Ren, Tsang Ing

doi:10.1109/ijcnn.2011.6033350

Cited by 27 publications

(35 citation statements)

References 30 publications

(31 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For each dataset, the experiments were conducted using 20 replications. For each replication, the datasets were divided using the holdout method [71] on the basis of 50% for training, 25% for the dynamic selection dataset, DSEL, and 25% for the test set, G. They were selected empirically based on previous publications [46,18,6]. Hence, the size of the meta-feature vector is 67 ((7 × 8) +5 + 6).…”

Section: Experimental Protocolmentioning

confidence: 99%

“…The same pool of classifiers is used for all techniques in order to ensure a fair comparison. For all techniques, the size of the region of competence, K, was set at 7 since it achieved the best result in previous experiments [46,6]. The results are shown in As shown in Figure 6, the selected meta-features vary considerably according to different classification problems.…”

Section: Comparison With the State-of-the-art Des Techniquesmentioning

confidence: 99%

See 1 more Smart Citation

META-DES.Oracle: Meta-learning and feature selection for dynamic ensemble selection

2017

View full text Add to dashboard Cite

Dynamic ensemble selection (DES) techniques work by estimating the competence level of each classifier from a pool of classifiers, and selecting only the most competent ones for the classification of a specific test sample. The key issue in DES is defining a suitable criterion for calculating the classifiers' competence. There are several criteria available to measure the level of competence of base classifiers, such as local accuracy estimates and ranking. However, using only one criterion may lead to a poor estimation of the classifier's competence. In order to deal with this issue, we have proposed a novel dynamic ensemble selection framework using meta-learning, called META-DES. A meta-classifier is trained, based on the meta-features extracted from the training data, to estimate the level of competence of a classifier for the classification of a given query sample. An important aspect of the META-DES framework is that multiple criteria can be embedded in the system encoded as different sets of meta-features. However, some DES criteria are not suitable for every classification problem. For instance, local accuracy estimates may produce poor results when there is a high degree of overlap between the classes. Moreover, a higher classification accuracy can be obtained if the performance of the meta-classifier is optimized for the corresponding data. In this paper, we propose a novel version of the META-DES framework based on the formal definition of the Oracle, called META-DES.Oracle. The Oracle is an abstract method that represents an ideal classifier selection scheme. A meta-feature selection scheme using an overfitting cautious Binary Particle Swarm Optimization (BPSO) is proposed for improving the performance of the meta-classifier. The difference between the outputs obtained by the meta-classifier and those presented by the Oracle is minimized. Thus, the meta-classifier is expected to obtain results that are similar to the Oracle. Experiments carried out using 30 classifi-1 arXiv:1811.00217v1 [cs.LG] 1 Nov 2018 cation problems demonstrate that the optimization procedure based on the Oracle definition leads to a significant improvement in classification accuracy when compared to previous versions of the META-DES framework and other state-of-the-art DES techniques.

show abstract

Section: Experimental Protocolmentioning

confidence: 99%

Section: Comparison With the State-of-the-art Des Techniquesmentioning

confidence: 99%

META-DES.Oracle: Meta-learning and feature selection for dynamic ensemble selection

2017

View full text Add to dashboard Cite

show abstract

“…NNF) works as a noise r class boundaries [25]. et ‫ܦ‬ (data set where the d) generating the data set hm 2.…”

Section: B Nearest Neighbor Filtermentioning

confidence: 99%

Optimizing Dynamic Ensemble Selection Procedure by Evolutionary Extreme Learning Machines and a Noise Reduction Filter

Lima

Ludermir

2013

2013 IEEE 25th International Conference on Tools With Artificial Intelligence

View full text Add to dashboard Cite

Ensemble of classifier is an effective way of improving performance of individual classifiers. However, the choice of the ensemble members can become a very difficult task, which, in some cases, can lead to ensembles with no performance improvement. Dynamic ensemble selection systems aim to select a group of classifiers that is most adequate for a specific query pattern. In this paper, we present a strategy that optimizes the dynamic ensemble selection procedure. Initially, a pool of classifiers has been built in an automatic way through an evolutionary algorithm. After, we improved the regions of competence in order to avoid noise and create smoother class boundaries. Finally, we use a dynamic ensemble selection rule. Extreme Learning Machines were used in the classification phase. Performance of the system was compared against other methods.

show abstract

“…Then, the selected classifier or EoC is used for the classification of all unseen test samples. In contrast, dynamic ensemble selection approaches (DES) [4,5,6,7,8,9,10,11,12,13] select a different classifier or a different EoC for each new test sample. DES techniques rely on the assumption that each base classifier is an expert in a different local region of the feature space [14].…”

Section: Introductionmentioning

confidence: 99%

“…Most DES techniques [4,12,11,10,15,16,17,18] use estimates of the classifiers' local accuracy in small regions of the feature space surrounding the query instance as a search criterion to perform the ensemble selection. However, in our previous work [10], we demonstrated that the use of local accuracy estimates alone is insufficient to achieve results close to the Oracle performance. The Oracle is an abstract model defined in [19] which always selects the classifier that predicted the correct label, for the given query sample, if such classifier exists.…”

Section: Introductionmentioning

confidence: 99%

META-DES: A dynamic ensemble selection framework using meta-learning

Cruz

Sabourin

Cavalcanti

et al. 2015

Pattern Recognition

Self Cite

205

129

View full text Add to dashboard Cite

Dynamic ensemble selection systems work by estimating the level of competence of each classifier from a pool of classifiers. Only the most competent ones are selected to classify a given test sample. This is achieved by defining a criterion to measure the level of competence of a base classifier, such as, its accuracy in local regions of the feature space around the query instance. However, using only one criterion about the behavior of a base classifier is not sufficient to accurately estimate its level of competence. In this paper, we present a novel dynamic ensemble selection framework using meta-learning. We propose five distinct sets of meta-features, each one corresponding to a different criterion to measure the level of competence of a classifier for the classification of input samples. The meta-features are extracted from the training data and used to train a meta-classifier to predict whether or not a base classifier is competent enough to classify an input instance. During the generalization phase, the meta-features are extracted from the query instance and passed down as input to the meta-classifier. The meta-classifier estimates, whether a base classifier is competent enough to be added to the ensemble. Experiments are conducted over several small sample size classification problems, i.e., problems with a high degree of uncertainty due to the lack of training data. Experimental results show the proposed meta-learning framework greatly improves classification accuracy when compared against current state-of-the-art dynamic ensemble selection techniques.

show abstract

A method for dynamic ensemble selection based on a filter and an adaptive distance to improve the quality of the regions of competence

Cited by 27 publications

References 30 publications

META-DES.Oracle: Meta-learning and feature selection for dynamic ensemble selection

META-DES.Oracle: Meta-learning and feature selection for dynamic ensemble selection

Optimizing Dynamic Ensemble Selection Procedure by Evolutionary Extreme Learning Machines and a Noise Reduction Filter

META-DES: A dynamic ensemble selection framework using meta-learning

Contact Info

Product

Resources

About