An Ensemble System with Random Projection and Dynamic Ensemble Selection

Intelligent Information and Database Systems

et al. 2020

Self Cite

Combining classifiers in an ensemble is beneficial in achieving better prediction than using a single classifier. Furthermore, each classifier can be associated with a weight in the aggregation to boost the performance of the ensemble system. In this work, we propose a novel dynamic weighted ensemble method. Based on the observation that each classifier provides a different level of confidence in its prediction, we propose to encode the level of confidence of a classifier by associating with each classifier a credibility threshold, computed from the entire training set by minimizing the entropy loss function with the mini-batch gradient descent method. On each test sample, we measure the confidence of each classifier's output and then compare it to the credibility threshold to determine whether a classifier should be attended in the aggregation. If the condition is satisfied, the confidence level and credibility threshold are used to compute the weight of contribution of the classifier in the aggregation. By this way, we are not only considering the presence but also the contribution of each classifier based on the confidence in its prediction on each test sample. The experiments conducted on a number of datasets show that the proposed method is better than some benchmark algorithms including a non-weighted ensemble method, two dynamic ensemble selection methods, and two Boosting methods.

Section: Methodsmentioning

confidence: 99%

Section: Background and Related Workmentioning

confidence: 99%

Section: Background and Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Confidence in Prediction: An Approach for Dynamic Weighted Ensemble

Thuan

Intelligent Information and Database Systems

et al. 2020

Self Cite

“…In detail, the Binary Relevance (BR) method learns on each chunk to obtain the ensemble of classifiers, and these classifiers" outputs are concatenated to the original feature space as the metadata which is learned by a second BR. For each test sample, the weights for the classifiers are determined via the dynamic classifier ensemble approach [35] i.e. based on the performance of the classifiers on the test sample"s neighbors in the latest chunk.…”

Section: Multi-label Classification For Data Streammentioning

confidence: 99%

Multi-label classification via label correlation and first order feature dependance in a data stream

et al. 2019

Pattern Recognition

Self Cite

A Homogeneous-Heterogeneous Ensemble of Classifiers

Communications in Computer and Information Science

et al. 2020

Self Cite

In this study, we introduce an ensemble system by combining homogeneous ensemble and heterogeneous ensemble into a single framework. Based on the observation that the projected data is significantly different from the original data as well as each other after using random projections, we construct the homogeneous module by applying random projections on the training data to obtain the new training sets. In the heterogeneous module, several learning algorithms will train on the new training sets to generate the base classifiers. We propose four combining algorithms based on Sum Rule and Majority Vote Rule for the proposed ensemble. Experiments on some popular datasets confirm that the proposed ensemble method is better than several well-known benchmark algorithms. proposed framework has great flexibility when applied to real-world applications. The proposed framework has great flexibility when applied to real-world applications by using any techniques that make rich training data for the homogeneous module, as well as using any set of learning algorithms for the heterogeneous module.