Hybrid Incremental Ensemble Learning for Noisy Real-World Data Classification

In this study, a novel framework to combine multiple classifiers in an ensemble system is introduced. Here we exploit the concept of information granule to construct granular prototypes for each class on the outputs of an ensemble of base classifiers. In the proposed method, uncertainty in the outputs of the base classifiers on training observations is captured by an interval-based representation. To predict the class label for a new observation, we first determine the distances between the output of the base classifiers for this observation and the class prototypes, then the predicted class label is obtained by choosing the label associated with the shortest distance. In the experimental study, we combine several learning algorithms to build the ensemble system and conduct experiments on the UCI, colon cancer, and selected CLEF2009 datasets. The experimental results demonstrate that the proposed framework outperforms several benchmarked algorithms including two trainable combining methods, i.e.

show abstract

“…There are also hybrid approaches to weigh base classifiers in Random Subspace [45], and weigh feature subspaces in Bagging [46]. Yu et al [47]…”

Section: Ensemble Methodsmentioning

confidence: 99%

Combining heterogeneous classifiers via granular prototypes

Nguyen

Pham

et al. 2018

Applied Soft Computing

View full text Add to dashboard Cite

show abstract

“…It only selects good base classifiers from the original ensemble model each time to predict new data and adds or deletes classifiers by demands. Z. Yu et al [21] proposed a hybrid incremental ensemble learning (HIEL) approach, which adopts Bagging algorithm to generate diverse ensemble members while using AdaBoost to evaluate the importance of instances, and incrementally change the weights of base classifiers. In [22], according to the classification performance and stochastic sensitivities, it adopts dynamic cost-sensitive weighting method to relieve concept drift.…”

Section: Related Workmentioning

confidence: 99%

An Incremental Learning Method Based on Dynamic Ensemble RVM for Intrusion Detection

Gao

Cui

et al. 2022

IEEE Trans. Netw. Serv. Manage.

View full text Add to dashboard Cite

Due to the dynamic changes of network data over time, static intrusion detection systems cannot adapt well to the behavioral characteristics of the input network data, resulting in reduced detection accuracy. In addition, continuous input data streams will bring huge challenges to resource storage and computing costs. Therefore, we propose an intrusion detection method of dynamic ensemble incremental learning (DEIL-RVM), and realize a dynamically adjusted ensemble intrusion detection model. In which a new overall misclassification probability weight value (OMPW) based on incremental set or data chunk is designed as the basis for updating the ensemble model, and it can be used to prune and replace the poor base component in the ensemble model. We presented a probabilistic decision function taking into account the posterior probability of each base RVM model dividing the sample into each category. The RVM with high sparsity is used as the base component to obtain the good balance between the accuracy, robustness and resource consumption, which can sacrifice less time and storage cost in ensemble incremental learning while achieving higher detection accuracy and stability in network data streams.

show abstract

“…The number of decision trees and the maximum tree depth are two important parameters for RF and ERF according to [6]. Therefore, they are developed as terminals, N T and M D. The values for N T is in the range of [50,500] with a step of 10 and the values for M D is in the range of [10,100] with a step of 10. The maximum values for N T and M D are set according to that in [6].…”

Section: New Terminal Setmentioning

confidence: 99%

Genetic Programming With a New Representation to Automatically Learn Features and Evolve Ensembles for Image Classification

Bi¹,

Xue²,

Zhang³

2020

Preprint

View full text Add to dashboard Cite

Image classification is a popular task in machine learning and computer vision, but it is very challenging due to high variation crossing images. Using ensemble methods for solving image classification can achieve higher classification performance than using a single classification algorithm. However, to obtain a good ensemble, the component (base) classifiers in an ensemble should be accurate and diverse. To solve image classification effectively, feature extraction is necessary to transform raw pixels into high-level informative features. However, this process often requires domain knowledge. This article proposes an evolutionary approach based on genetic programming to automatically and simultaneously learn informative features and evolve effective ensembles for image classification. The new approach takes raw images as inputs and returns predictions of class labels based on the evolved classifiers. To achieve this, a new individual representation, a new function set, and a new terminal set are developed to allow the new approach to effectively find the best solution. More important, the solutions of the new approach can extract informative features from raw images and can automatically address the diversity issue of the ensembles. In addition, the new approach can automatically select and optimize the parameters for the classification algorithms in the ensemble. The performance of the new approach is examined on 13 different image classification datasets of varying difficulty and compared with a large number of effective methods. The results show that the new approach achieves better classification accuracy on most datasets than the competitive methods. Further analysis demonstrates that the new approach can evolve solutions with high accuracy and diversity.

show abstract

Hybrid Incremental Ensemble Learning for Noisy Real-World Data Classification

Cited by 38 publications

References 67 publications

Combining heterogeneous classifiers via granular prototypes

Combining heterogeneous classifiers via granular prototypes

An Incremental Learning Method Based on Dynamic Ensemble RVM for Intrusion Detection

Genetic Programming With a New Representation to Automatically Learn Features and Evolve Ensembles for Image Classification

Contact Info

Product

Resources

About