Multi-layer heterogeneous ensemble with classifier and feature selection

Nguyen, Tien Thanh; Pham, Nang Van; Dang, Manh Truong; Luong, Anh Vu; McCall, John; Liew, Alan Wee‐Chung

doi:10.1145/3377930.3389832

Cited by 13 publications

(9 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…After that, several deep ensemble systems were introduced such as deep ensemble models of incremental classifiers [7], an ensemble of SVM classifiers with AdaBoost in finding model parameters [17], and deep ensemble models for multi-label learning [21]. Nguyen et al [15] proposed MULES, a deep ensemble system with classifier and feature selection in each layer. The optimization problem was considered under bi-objectives: maximizing classification accuracy and diversity of EoC in each layer.…”

Section: Ensemble Learning and Ensemble Selectionmentioning

confidence: 99%

“…All these methods were constructed by using 200 learners. Three deep learning models were compared with VEGAS: gcForest (4 forests with 200 trees in each forest) [22], MULES [15], and Multiple Layer Perceptron (MLP). For MULES, Generate a random number rc ∈ [0, 1]…”

Section: Experimental Settingsmentioning

confidence: 99%

“…The introduction of gcForest has shown that DNN is only a subset of deep models or deep learning models can be constructed with multiple layers of non-differentiable learning modules. Experiments on some popular datasets have shown that deep ensemble models outperform not only DNNs but also several state-of-the-art ensemble methods [22], [15].…”

Section: Introductionmentioning

confidence: 99%

“…Calculate selection probabilities by (7) 27: end for 28: Return individual (encoding and associated deep model) with the best fitness from the last generation we used parameter settings like in the original paper [15]. It is noted that the performance of MLP significantly depends on the network structure.…”

mentioning

confidence: 99%

“…On the Tic-Tac-Toe dataset, VEGAS takes only 0.016 second for classification with 4 layers and 6 classifiers in total. Meanwhile, gcForest (6 layers and 4800 classifiers in total) used 0.62 second to classify all test instances, and MULES (4 layers with 11 classifiers in total) used 0.26 second with its selected configuration [15].…”

mentioning

confidence: 99%

See 4 more Smart Citations

VEGAS: A Variable Length-Based Genetic Algorithm for Ensemble Selection in Deep Ensemble Learning

Han

Pham

et al. 2021

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

In this study, we introduce an ensemble selection method for deep ensemble systems called VEGAS. The deep ensemble models include multiple layers of the ensemble of classifiers (EoC). At each layer, we train the EoC and generates training data for the next layer by concatenating the predictions for training observations and the original training data. The predictions of the classifiers in the last layer are combined by a combining method to obtain the final collaborated prediction. We further improve the prediction accuracy of a deep ensemble model by searching for its optimal configuration, i.e., the optimal set of classifiers in each layer. The optimal configuration is obtained using the Variable-Length Genetic Algorithm (VLGA) to maximize the prediction accuracy of the deep ensemble model on the validation set. We developed three operators of VLGA: roulette wheel selection for breeding, a chunk-based crossover based on the number of classifiers to generate new offsprings, and multiple random points-based mutation on each offspring. The experiments on 20 datasets show that VEGAS outperforms selected benchmark algorithms, including two well-known ensemble methods (Random Forest and XgBoost) and three deep learning methods (Multiple Layer Perceptron, gcForest, and MULES).

show abstract

Section: Ensemble Learning and Ensemble Selectionmentioning

confidence: 99%

Section: Experimental Settingsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

VEGAS: A Variable Length-Based Genetic Algorithm for Ensemble Selection in Deep Ensemble Learning

Han

Pham

et al. 2021

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

Two-layer Ensemble of Deep Learning Models for Medical Image Segmentation

Dang,

Nguyen,

McCall

et al. 2024

Cogn Comput

View full text Add to dashboard Cite

One of the most important areas in medical image analysis is segmentation, in which raw image data is partitioned into structured and meaningful regions to gain further insights. By using Deep Neural Networks (DNN), AI-based automated segmentation algorithms can potentially assist physicians with more effective imaging-based diagnoses. However, since it is difficult to acquire high-quality ground truths for medical images and DNN hyperparameters require significant manual tuning, the results by DNN-based medical models might be limited. A potential solution is to combine multiple DNN models using ensemble learning. We propose a two-layer ensemble of deep learning models in which the prediction of each training image pixel made by each model in the first layer is used as the augmented data of the training image for the second layer of the ensemble. The prediction of the second layer is then combined by using a weight-based scheme which is found by solving linear regression problems. To the best of our knowledge, our paper is the first work which proposes a two-layer ensemble of deep learning models with an augmented data technique in medical image segmentation. Experiments conducted on five different medical image datasets for diverse segmentation tasks show that proposed method achieves better results in terms of several performance metrics compared to some well-known benchmark algorithms. Our proposed two-layer ensemble of deep learning models for segmentation of medical images shows effectiveness compared to several benchmark algorithms. The research can be expanded in several directions like image classification.

show abstract