A Novel 2-Stage Combining Classifier Model with Stacking and Genetic Algorithm Based Feature Selection

Nguyen, Tien Thanh; Liew, Alan Wee‐Chung; Pham, Xuan Cuong; Nguyen, Mai P.

doi:10.1007/978-3-319-09339-0_4

Cited by 10 publications

(12 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…classifiers to obtain better result than using each individual learner. In general, ensemble learning can be categorised into two main types namely homogeneous ensemble and heterogeneous ensemble [14], [15]. In homogeneous ensemble, many new training sets are generated from the original training data.…”

Section: B Ensemble Methods For Text Classificationmentioning

confidence: 99%

WEC: Weighted Ensemble of Text Classifiers

Upadhyay

Nguyen

Massie

et al. 2020

2020 IEEE Congress on Evolutionary Computation (CEC)

Self Cite

View full text Add to dashboard Cite

Text classification is one of the most important tasks in the field of Natural Language Processing. There are many approaches that focus on two main aspects: generating an effective representation; and selecting and refining algorithms to build the classification model. Traditional machine learning methods represent documents in vector space using features such as term frequencies, which have limitations in handling the order and semantics of words. Meanwhile, although achieving many successes, deep learning classifiers require substantial resources in terms of labelled data and computational complexity. In this work, a weighted ensemble of classifiers (WEC) is introduced to address the text classification problem. Instead of using majority vote as the combining method, we propose to associate each classifier's prediction with a different weight when combining classifiers. The optimal weights are obtained by minimising a loss function on the training data with the Particle Swarm Optimisation algorithm. We conducted experiments on 5 popular datasets and report classification performance of algorithms with classification accuracy and macro F1 score. WEC was run with several different combinations of traditional machine learning and deep learning classifiers to show its flexibility and robustness. Experimental results confirm the advantage of WEC, especially on smaller datasets.

show abstract

Section: B Ensemble Methods For Text Classificationmentioning

confidence: 99%

WEC: Weighted Ensemble of Text Classifiers

Upadhyay

Nguyen

Massie

et al. 2020

2020 IEEE Congress on Evolutionary Computation (CEC)

Self Cite

View full text Add to dashboard Cite

show abstract

“…In detail, Nguyen et al [10] encoded the base classifiers and the features in a single chromosome and used Genetic Algorithm to simultaneously search for the optimal set of classifiers and associated features. Nguyen et al [9] also proposed a new encoding for meta-data feature and used Genetic Algorithm to search for the optimal set of meta-data features for the Decision Tree meta-classifiers. Shunmugapriya and Kanmani [16] used Artificial Bee Colony (ABC) to find the optimal set of base classifiers and the meta-classifiers.…”

Section: Heterogeneous Ensemble Methodsmentioning

confidence: 99%

“…The Optimal Decision Templates of Fertility and Hayes-Roth datasets -Random Subspace[1]: We used Decision Tree as the learning algorithm to train 200 base classifiers. -GA Meta-data[9]: The method searches for the optimal subset of meta-data for heterogeneous ensemble.…”

mentioning

confidence: 99%

Evolving an Optimal Decision Template for Combining Classifiers

Nguyen

Luong

Dang

et al. 2019

Neural Information Processing

Self Cite

View full text Add to dashboard Cite

In this paper, we aim to develop an effective combining algorithm for ensemble learning systems. The Decision Template method, one of the most popular combining algorithms for ensemble systems, does not perform well when working on certain datasets like those having imbalanced data. Moreover, point estimation by computing the average value on the outputs of base classifiers in the Decision Template method is sometimes not a good representation, especially for skewed datasets. Here we propose to search for an optimal decision template in the combining algorithm for a heterogeneous ensemble. To do this, we first generate the base classifier by training the pre-selected learning algorithms on the given training set. The meta-data of the training set is then generated via cross validation. Using the Artificial Bee Colony algorithm, we search for the optimal template that minimizes the empirical 0-1 loss function on the training set. The class label is assigned to the unlabeled sample based on the maximum of the similarity between the optimal decision template and the sample's meta-data. Experiments conducted on the UCI datasets demonstrated the superiority of the proposed method over several benchmark algorithms.

show abstract

“…Moreover, the estimate of the joint distribution is obtained by aggregating multiple models associated with each features. Since many research [52][53][54][55][56][57][58][59][60] have shown that aggregating multiple models can improve the classification accuracy, the 1dependence method is expected to enhance the performance of the system.…”

Section: Accepted Manuscriptmentioning

confidence: 99%

“…A N U S C R I P T 27 multiple models in an ensemble system can usually improve the classification accuracy [52][53][54][55][56][57][58][59][60].…”

Section: A C C E P T E D Mmentioning

confidence: 99%