MtL-NFW: A Meta-Learning Framework for Automated Noise Filter Selection and Hyperparameter Optimization in Auto-ML

Abstract

The extensive implementation of machine learning (ML) has transformed data analysis and decision-making processes. However, the process of choosing suitable ML algorithms for a specific task remains challenging, commonly referred to as the Algorithm Selection Problem (ASP) in academic literature. Automated machine learning (Auto-ML) tackles algorithm selection and hyperparameter adjustments by automating these processes, hence decreasing the required time and expertise. The current study is focused on the automated selection of noise filters for the task of classifying data with noisy labels. Class label noise, resulting from erroneous data labels, has a greater effect on the learning process compared to attribute-level noise. The existing Auto-ML approaches for selecting suitable noise filters overlook hyperparameter adjustment, which is important for the effectiveness of noise filters. Instead, it relies on computationally demanding techniques such as grid search. In addition, the current Auto-ML algorithms utilize random noise imputation techniques in their methodologies that are not 1 able to adequately capture the complex noise distributions typically present in real-world datasets. In order to overcome these limitations, we propose the introduction of Meta-learning for Noise-Filter Workflow Selection (MtL-NFW), an innovative framework that automate and integrate adequate hyperparameter selection along with noise filter for a certain task. This eliminates the need to rely on manual and computationally intensive methods for hyperparameters settings. Our second contribution is to leverage and employ the Neighbor-wise noise imputation technique to our framework, that induce noise based on proximity to class boundaries, generating more robust meta-training data. Our framework evaluates 114 classification datasets with six noise filtering algorithms and Random Forest as the base classifier, extracting six groups of meta-features and generating a meta-knowledge base of 51,642 configurations. XGBoost trained as meta-learner, predicts and recommends optimal noise filter and hyperparameter combinations for new datasets. Empirical validation on 114 OpenML datasets shows that MtL-NFW significantly improves the accuracy and robustness of noise filter recommendations, outperforming four state-of-the-art Auto-ML methods.