Automatic Recommendation Method for Classifier Ensemble Structure Using Meta-Learning

Silva, Robercy Alves da; Canuto, Anne M. P.; Barreto, Cephas Alves da Silveira; Xavier-Júnior, João C.

doi:10.1109/access.2021.3099689

Cited by 7 publications

(4 citation statements)

References 52 publications

(56 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Details of each subcomponent in the architecture can be found in the next subsections. There are two fundamental prerequisites for applying ensemble learning, namely: (1) The base models should be accurate. 2.…”

Section: Architecture Of Autofe-selmentioning

confidence: 99%

“…Data analysis is a multistep process that employs algorithms for each step, such as preprocessing like data labeling, cleaning, handling imbalanced classes, and feature selection before training a base model on the data. The given diversity of data analysis tasks and large number of available ML algorithms pose a significant limitation, i.e., how to select adequate algorithms for a given problem from the large set of available candidate algorithms [1]. The process of choosing an adequate algorithm for each step of the multistep process of data analysis is an iterative, and nontrivial task, formally known as the "algorithm selection problem" (ASP) in literature [2].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

AutoFe-Sel: A Meta-learning based methodology for Recommending Feature Subset Selection Algorithms

2023

KSII TIIS

View full text Add to dashboard Cite

Automated machine learning, often referred to as "AutoML," is the process of automating the time-consuming and iterative procedures that are associated with the building of machine learning models. There have been significant contributions in this area across a number of different stages of accomplishing a data-mining task, including model selection, hyperparameter optimization, and preprocessing method selection. Among them, preprocessing method selection is a relatively new and fast growing research area. The current work is focused on the recommendation of preprocessing methods, i.e., feature subset selection (FSS) algorithms. One limitation in the existing studies regarding FSS algorithm recommendation is the use of a single learner for meta-modeling, which restricts its capabilities in the metamodeling. Moreover, the meta-modeling in the existing studies is typically based on a single group of data characterization measures (DCMs). Nonetheless, there are a number of complementary DCM groups, and their combination will allow them to leverage their diversity, resulting in improved meta-modeling. This study aims to address these limitations by proposing an architecture for preprocess method selection that uses ensemble learning for meta-modeling, namely AutoFE-Sel. To evaluate the proposed method, we performed an extensive experimental evaluation involving 8 FSS algorithms, 3 groups of DCMs, and 125 datasets. Results show that the proposed method achieves better performance compared to three baseline methods. The proposed architecture can also be easily extended to other preprocessing method selections, e.g., noise-filter selection and imbalance handling method selection.

show abstract

Section: Architecture Of Autofe-selmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

AutoFe-Sel: A Meta-learning based methodology for Recommending Feature Subset Selection Algorithms

2023

KSII TIIS

View full text Add to dashboard Cite

show abstract

“…According to the base classifiers' type [27][28][29], ensemble learning contains a homogeneous ensemble and heterogeneous ensemble. The heterogeneous ensemble has an advantage in improving the accuracy and generalization level of the classification model.…”

Section: Ensemble Learningmentioning

confidence: 99%

Situation Assessment of Air Traffic Based on Complex Network Theory and Ensemble Learning

Liu,

Li,

Wen

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

With the rapid development of the air transportation industry, the air traffic situation is becoming more and more complicated. Determining the situation of air traffic is of great significance to ensure the safety and smoothness of air traffic. The strong subjectivity of assessment criteria, inaccurate assessment results and weak systemic assessment method are the main problems in air traffic situation assessment research. The aim of our research is to present an objective and accurate situation assessment method for air traffic systems. The paper presents a model to assess air traffic situation based on the complex network theory and ensemble learning. The air traffic weighted network model was introduced to systematically describe the real state of an air traffic system. Assessment criteria based on the complex network analysis method can systematically reflect the operational state of an air traffic weighted network system. We transformed the air traffic situation assessment into a binary classification, which makes situation assessment objective and accurate. Ensemble learning was introduced to improve the classification accuracy, which further improves the accuracy of the situation assessment model. The model was trained and tested on the dataset of the East China air traffic weighted network in 2019. Its average classification accuracy is 0.98. The recall and precision rates both exceed 0.95. Experiments have confirmed that the situation assessment model can accurately output air traffic situation value and situation level. Furthermore, the assessment results are consistent with the real operational situation of the air traffic in East China.

show abstract

“…Automated Machine Learning (Auto-ML) is a sub field of Machine Learning that focuses on automating the entire ML pipeline, from selecting algorithms and hyperparameters to data pre-processing approaches, in order to address ASP [2,3]. Auto-ML systems provide recommendations for the most appropriate algorithms for a given dataset, greatly decreasing the effort and knowledge needed to select and apply appropriate machine learning algorithms.…”

Section: Introductionmentioning

confidence: 99%

MtL-NFW: A Meta-Learning Framework for Automated Noise Filter Selection and Hyperparameter Optimization in Auto-ML

Khan,

Zhang,

Kumar

et al. 2024

Preprint

View full text Add to dashboard Cite

The extensive implementation of machine learning (ML) has transformed data analysis and decision-making processes. However, the process of choosing suitable ML algorithms for a specific task remains challenging, commonly referred to as the Algorithm Selection Problem (ASP) in academic literature. Automated machine learning (Auto-ML) tackles algorithm selection and hyperparameter adjustments by automating these processes, hence decreasing the required time and expertise. The current study is focused on the automated selection of noise filters for the task of classifying data with noisy labels. Class label noise, resulting from erroneous data labels, has a greater effect on the learning process compared to attribute-level noise. The existing Auto-ML approaches for selecting suitable noise filters overlook hyperparameter adjustment, which is important for the effectiveness of noise filters. Instead, it relies on computationally demanding techniques such as grid search. In addition, the current Auto-ML algorithms utilize random noise imputation techniques in their methodologies that are not 1 able to adequately capture the complex noise distributions typically present in real-world datasets. In order to overcome these limitations, we propose the introduction of Meta-learning for Noise-Filter Workflow Selection (MtL-NFW), an innovative framework that automate and integrate adequate hyperparameter selection along with noise filter for a certain task. This eliminates the need to rely on manual and computationally intensive methods for hyperparameters settings. Our second contribution is to leverage and employ the Neighbor-wise noise imputation technique to our framework, that induce noise based on proximity to class boundaries, generating more robust meta-training data. Our framework evaluates 114 classification datasets with six noise filtering algorithms and Random Forest as the base classifier, extracting six groups of meta-features and generating a meta-knowledge base of 51,642 configurations. XGBoost trained as meta-learner, predicts and recommends optimal noise filter and hyperparameter combinations for new datasets. Empirical validation on 114 OpenML datasets shows that MtL-NFW significantly improves the accuracy and robustness of noise filter recommendations, outperforming four state-of-the-art Auto-ML methods.

show abstract

Automatic Recommendation Method for Classifier Ensemble Structure Using Meta-Learning

Cited by 7 publications

References 52 publications

AutoFe-Sel: A Meta-learning based methodology for Recommending Feature Subset Selection Algorithms

AutoFe-Sel: A Meta-learning based methodology for Recommending Feature Subset Selection Algorithms

Situation Assessment of Air Traffic Based on Complex Network Theory and Ensemble Learning

MtL-NFW: A Meta-Learning Framework for Automated Noise Filter Selection and Hyperparameter Optimization in Auto-ML

Contact Info

Product

Resources

About