Margin-Based Pareto Ensemble Pruning: An Ensemble Pruning Algorithm That Learns to Search Optimized Ensembles

Abstract

The ensemble pruning system is an effective machine learning framework that combines several learners as experts to classify a test set. Generally, ensemble pruning systems aim to define a region of competence based on the validation set to select the most competent ensembles from the ensemble pool with respect to the test set. However, the size of the ensemble pool is usually fixed, and the performance of an ensemble pool heavily depends on the definition of the region of competence. In this paper, a dynamic pruning framework called margin-based Pareto ensemble pruning is proposed for ensemble pruning systems. The framework explores the optimized ensemble pool size during the overproduction stage and finetunes the experts during the pruning stage. The Pareto optimization algorithm is used to explore the size of the overproduction ensemble pool that can result in better performance. Considering the information entropy of the learners in the indecision region, the marginal criterion for each learner in the ensemble pool is calculated using margin criterion pruning, which prunes the experts with respect to the test set. The effectiveness of the proposed method for classification tasks is assessed using datasets. The results show that margin-based Pareto ensemble pruning can achieve smaller ensemble sizes and better classification performance in most datasets when compared with state-of-the-art models.

Development and Internal Validation of a Nomogram to Predict Mortality During the ICU Stay of Thoracic Fracture Patients Without Neurological Compromise: An Analysis of the MIMIC-III Clinical Database

“…With the rapid development of computer technology in recent years, the combination of machine learning technology and medical practice has become a major trend ( 19 – 21 , 26 , 27 ). Along with the continuous innovation of convolutional algorithms, from LeNet by Lecun et al ( 36 ) to ResNet by He et al ( 37 ), computer-aided decision making through imaging has become a hot topic in medical research, such as prediction of BMI by facial image features to predict BMI ( 38 ) and fundus images to predict diabetic retinopathy ( 39 ).…”

“…Then, the least absolute shrinkage and selection operator (LASSO) expression was used for screening predictors of mortality ( 25 ). For the cross-validation results, we selected lambda = min to determine the final candidate variables ( 26 , 27 ). The multivariate logistic regression model was established using these variables, and multicollinearity was evaluated by variable inflation factors (VIF).…”

Ensemble Pruning via Quadratic Margin Maximization

“…Other selection-based methods include formulating the selection of the pruned sub-sensemble as an integer optimization heuristic (see e.g., [20]). Recently, combinations of margin-optimizing and diversity pruning methods have been of special interest to researchers (see e.g., [26,[73][74][75][76][77]). The list of selectionbased pruning methods presented here is not exhaustive.…”

Innovation and Development of Cultural and Creative Industries Based on Big Data for Industry 5.0

“…From the Perspective of the Organizational Structure of the Industry. Industry in the traditional sense generally refers to similar enterprises organizing production in accordance with the general processes, lacking horizontal connections with each other, as well as industrial organization forms often reflecting the characteristics of vertical integration [28] and forming a new consumer market. e same or different industries converge and develop, and the industrial clusters that provide creative services to all industries have broken the boundaries of traditional industries in organizational form [29].…”