Multi-label classification with weighted classifier selection and stacked ensemble

Abstract

Multi-label classification has attracted increasing attention for use in various application scenarios, such as medical diagnosis and semantic annotation. A large number of algorithms have been proposed for multi-label classification where many are ensemble-based. However, these ensemble-based methods usually employ bagging schemes for ensemble construction, with comparatively few stacked ensembles for multilabel classification. Existing research on stacked ensemble schemes remains active, but several issues remain such as (1) little has been done to learn the weights of classifiers for combined classifier selection; (2) pairwise label correlations is not investigated sufficiently to improve classification performance. To address these issues, we propose a novel approach that simultaneously exploits label correlations and the process of learning classifier weights to improve the existing stacked ensemble schemes. First, we introduce a weighted stacked ensemble for multi-label classification and use sparsity for regularization to facilitate classifier selection and ensemble construction. Second, we consider pairwise label correlations for assigning high similar weights to improve the classification performance. Finally, we develop an optimization algorithm based on the accelerated proximal gradient and the block coordinate descent techniques to find the optimal solution efficiently. Extensive experiments on publicly available datasets and real Cardiovascular and Cerebrovascular Disease datasets demonstrate that our proposed algorithm outperforms related state-of-the-art methods.

On the Cost-Effectiveness of Stacking of Neural and Non-Neural Methods for Text Classification: Scenarios and Performance Prediction

“…With the saved models and the predictions in the validation set, a metalayer (another learning algorithm) is used to learn how to combine the predictions in the combination. Recent work reported high effectiveness with stacking for multiple ATC tasks, such as topic classification (Campos et al, 2017;Abuhaiba and Dawoud, 2017), sentiment analysis (Carvalho and Plastino, 2020;Onan et al, 2016) and multi-label classification (Xia et al, 2020;Weng et al, 2019). Particularly, stacking provided substantial effectiveness improvements on recently proposed decisiontree-based algorithms (Campos et al, 2017) and with methods trained on different representations (including word embeddings) (Carvalho and Plastino, 2020;Pelle et al, 2018;Onan et al, 2016).…”

Evolutionary ensemble feature selection learning for image‐based assessment of lymphedema arm volume

“…There are different ensemble learning techniques in literature, which aggregates the results of several base learners to achieve the final decision 41 . These techniques can be categorized according to the type of base learners into (1) ensemble of the same base learners with different parameters (e.g., different training samples), 42 and (2) ensemble of the different base learners 43 . There are also some hybrid ensemble learning and feature selection methods, 44‐48 which utilized a feature selection algorithm to select an appropriate feature subset for the ensemble learning model.…”

Classification using link prediction

“…MLWSE [18] is a stacked ensemble algorithm which uses pairwise label correlations and it uses label-specific meta features to facilitate the classification process. In this algorithm, two classifiers with highly correlated labels share high similar weights as opposed to two classifiers with weakly correlated labels.…”