Dynamic Ensemble Selection for Imbalanced Data Streams With Concept Drift

Jiao, Botao; Guo, Yinan; Gong, Dunwei; Chen, Qiuju

doi:10.1109/tnnls.2022.3183120

Cited by 33 publications

(26 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, Nguyen et al [59] proposed a selection strategy that combines aspects of both static and dynamic ensemble selection. Other recently proposed dynamic ensemble selection methods can be found in [64]- [66]. Meanwhile, Pérez-Gállego et al [67] present a detailed description of ensemble selection strategies and a comparison between static and dynamic selection methods.…”

Section: B Ensemble Selectionmentioning

confidence: 99%

A Survey of Ensemble Learning: Concepts, Algorithms, Applications, and Prospects

2022

View full text Add to dashboard Cite

Ensemble learning techniques have achieved state-of-the-art performance in diverse machine learning applications by combining the predictions from two or more base models. This paper presents a concise overview of ensemble learning, covering the three main ensemble methods: bagging, boosting, and stacking, their early development to the recent state-of-the-art algorithms. The study focuses on the widely used ensemble algorithms, including random forest, adaptive boosting (AdaBoost), gradient boosting, extreme gradient boosting (XGBoost), light gradient boosting machine (LightGBM), and categorical boosting (CatBoost). An attempt is made to concisely cover their mathematical and algorithmic representations, which is lacking in the existing literature and would be beneficial to machine learning researchers and practitioners.

show abstract

Section: B Ensemble Selectionmentioning

confidence: 99%

A Survey of Ensemble Learning: Concepts, Algorithms, Applications, and Prospects

2022

View full text Add to dashboard Cite

show abstract

“…Class distributions are generally imbalanced in real-world data, as IEEE TRANSACTIONS ON ARTIFICIAL INTELLIGENCE certain objects or patterns appear more frequently. Various works have been done to tackle the imbalanced distribution in recent years, which raises the importance of evaluating deep learning algorithms on imbalanced datasets [29][30][31][32]. To study the effectiveness of the proposed method for classifying imbalanced image datasets, we adopted one synthetic imbalanced dataset based on CIFAR10 and one real-world dataset, the SVHN dataset.…”

Section: ) Classification Performance On Imbalanced Datasetsmentioning

confidence: 99%

Supervised Local Training With Backward Links for Deep Neural Networks

Guo

Fouda

Eltawil

et al. 2024

IEEE Trans. Artif. Intell.

View full text Add to dashboard Cite

The restricted training pattern in the standard BP requires end-to-end error propagation, causing large memory costs and prohibiting model parallelization. Existing local training methods aim to resolve the training obstacles by completely cutting off the backward path between modules and isolating their gradients. These methods prevent information exchange between modules and result in inferior performance. This work proposes a novel local training algorithm, BackLink, which introduces intermodule backward dependency and facilitates information to flow backward along with the network. To preserve the computational advantage of local training, BackLink restricts the error propagation length within the module. Extensive experiments performed in various deep convolutional neural networks demonstrate that our method consistently improves the classification performance of local training algorithms over other methods. For example, our method can surpass the conventional greedy local training method by 6.45% in accuracy in ResNet32 classifying CIFAR100 and recent work by 2.58% in ResNet110 classifying STL-10 with much lower complexity, respectively. Analysis of computational costs reveals that small overheads are incurred in GPU memory costs and runtime on multiple GPUs. Our method can lead up to a 79% reduction in memory cost and 52% in simulation runtime in ResNet110 compared to the standard BP. Therefore, our method could create new opportunities for improving training algorithms towards better efficiency for real-time learning applications. Impact Statement-Deep learning models are generally bulky with abundant parameters, demanding long training time and excessive computing power. It poses a great challenge for deploying these models in real-world applications. This study looks into the intrinsic limitation of the standard training algorithm. The proposed method increases information flow in the algorithm and improves image classification accuracy over conventional methods. It can substantially reduce memory cost and hardware runtime by 50%. With the proposed method, an intelligent system can be readily deployed on mobile devices and function to meet practical demands. It also enables computing systems to reduce excessive energy waste and cut down carbon footprints for deep learning applications, such as traffic control and autonomous driving.

show abstract

“…Meanwhile, in applications such as detecting abnormal events and diagnosing diseases, the importance of identifying the minority class is more or at least the same as that of the majority class. Two common approaches to manage imbalance in data streams are balancing the training data by manipulating it [29–31] or using imbalance‐resistant metrics to evaluate the quality of the base classifiers [28, 32]. In this research, we aimed to present a chunk‐based ensemble classification method for imbalanced data streams that is accurate for both minority and majority classes and can update itself when the concept changes.…”

Section: Problem Statementmentioning

confidence: 99%

“…A combination of oversampling and undersampling techniques can also be used to address class imbalance. Some of the ensemble algorithms that work at the data level are Imbalanced Data Stream Learner (IDSL) [29], Weighted Ensemble with one-class Classification and Over-sampling and Instance selection (WECOI) [31], and Dynamic Ensemble Selection for Imbalanced data streams with Concept Drift (DESICD) [30]. IDSL uses the k-nearest neighbor technique for oversampling the minority class to balance the distribution of training samples.…”

Section: Literature Reviewmentioning

confidence: 99%

Rarity updated ensemble with oversampling: An ensemble approach to classification of imbalanced data streams

Nouri,

Kiani,

Fadishei

2024

Statistical Analysis

View full text Add to dashboard Cite

Today's ever‐increasing generation of streaming data demands novel data mining approaches tailored to mining dynamic data streams. Data streams are non‐static in nature, continuously generated, and endless. They often suffer from class imbalance and undergo temporal drift. To address the classification of consecutive data instances within imbalanced data streams, this research introduces a new ensemble classification algorithm called Rarity Updated Ensemble with Oversampling (RUEO). The RUEO approach is specifically designed to exhibit robustness against class imbalance by incorporating an imbalance‐specific criterion to assess the efficacy of the base classifiers and employing an oversampling technique to reduce the imbalance in the training data. The RUEO algorithm was evaluated on a set of 20 data streams and compared against 14 baseline algorithms. On average, the proposed RUEO algorithm achieves an average‐accuracy of 0.69 on the real‐world data streams, while the chunk‐based algorithms AWE, AUE, and KUE achieve average‐accuracies of 0.48, 0.65, and 0.66, respectively. The statistical analysis, conducted using the Wilcoxon test, reveals a statistically significant improvement in average‐accuracy for the proposed RUEO algorithm when compared to 12 out of the 14 baseline algorithms. The source code and experimental results of this research work will be publicly available at https://github.com/vkiani/RUEO.

show abstract

Dynamic Ensemble Selection for Imbalanced Data Streams With Concept Drift

Cited by 33 publications

References 47 publications

A Survey of Ensemble Learning: Concepts, Algorithms, Applications, and Prospects

A Survey of Ensemble Learning: Concepts, Algorithms, Applications, and Prospects

Supervised Local Training With Backward Links for Deep Neural Networks

Rarity updated ensemble with oversampling: An ensemble approach to classification of imbalanced data streams

Contact Info

Product

Resources

About