MEGA: Predicting the best classifier combination using meta-learning and a genetic algorithm

Abstract

Classifier combination through ensemble systems is one of the most effective approaches to improve the accuracy of classification systems. Ensemble systems are generally used to combine classifiers; However, selecting the best combination of individual classifiers is a challenging task. In this paper, we propose an efficient assembling method that employs both meta-learning and a genetic algorithm for the selection of the best classifiers. Our method is called MEGA, standing for using MEta-learning and a Genetic Algorithm for algorithm recommendation. MEGA has three main components: Training, Model Interpretation and Testing. The Training component extracts meta-features of each training dataset and uses a genetic algorithm to discover the best classifier combination. The Model Interpretation component interprets the relationships between meta-features and classifiers using a priori and multi-label decision tree algorithms. Finally, the Testing component uses a weighted k-nearest-neighbors algorithm to predict the best combination of classifiers for unseen datasets. We present extensive experimental results that demonstrate the performance of MEGA. MEGA achieves superior results in a comparison of three other methods and, most importantly, is able to find novel interpretable rules that can be used to select the best combination of classifiers for an unseen dataset.

Explainable Rules and Heuristics in AI Algorithm Recommendation Approaches—A Systematic Literature Review and Mapping Study

“…4). The first, meta-learning, is employed by 4 solutions [25][26][27]30]. This methodology carries out the recommendation process by training another AI model (meta-learner) to learn from dataset characteristics (meta-features) and selecting the best performant AI algorithm (meta-target).…”

“…Similarly, authors in [30] presented a combination of meta-learning and a genetic algorithm to recommend the best classifier combinations. Their method, namely MEGA, extracts meta-features from datasets and discovers the best classifiers through a genetic algorithm.…”

“…3. From the nine selected papers, two of them belong to the early 2000s period [25,26], while the remaining belong to the last six years interval [27][28][29][30][31][32][33].…”

“…As can be seen in Fig. 5, the target AI algorithms of the recommendation process are mostly framed in the "classification" category [25,26,28,[30][31][32]. But there are some papers that also provide heuristics for clustering [28], regression [28,32], and feature selection problems [27,33].…”

“…In [30], authors collected a total of 60 meta-features that were used in previous works. These metafeatures were finally screened based on their number of citations in the literature, obtaining the final set displayed in Table 9.…”