A Literature Survey and Empirical Study of Meta-Learning for Classifier Selection

Abstract

Classification is the key and most widely studied paradigm in machine learning community. The selection of appropriate classification algorithm for a particular problem is a challenging task, formally known as algorithm selection problem (ASP) in literature. It is increasingly becoming focus of research in machine learning community. Meta-learning has demonstrated substantial success in solving ASP, especially in the domain of classification. Considerable progress has been made in classification algorithm recommendation and researchers have proposed various methods in literature that tackles ASP in many different ways in meta-learning setup. Yet there is a lack of survey and comparative study that critically analyze, summarize and assess the performance of existing methods. To fill these gaps, in this paper we first present a literature survey of classification algorithm recommendation methods. The survey shed light on the motivational reasons for pursuing classifier selection through meta-learning and comprehensively discusses the different phases of classifier selection based on a generic framework that is formed as an outcome of reviewing prior works. Subsequently, we critically analyzed and summarized the existing studies from the literature in three important dimensions i.e., meta-features, meta-learner and meta-target. In the second part of this paper, we present extensive comparative evaluation of all the prominent methods for classifier selection based on 17 classification algorithms and 84 benchmark datasets. The comparative study quantitatively assesses the performance of classifier selection methods and highlight the limitations and strengths of meta-features, meta-learners and meta-target in classification algorithm recommendation system. Finally, we conclude this paper by identifying current challenges and suggesting future work directions. We expect that this work will provide baseline and a solid overview of state of the art works in this domain to new researchers, and will steer future research in this direction.

Superpixel-Based Regional-Scale Grassland Community Classification Using Genetic Programming with Sentinel-1 SAR and Sentinel-2 Multispectral Images

“…In this study, we used Sentinel-2 multispectral and Sentinel-1 SAR imagery with large swath width to accommodate regional-scale studies. However, due to the spectral and spatial resolutions of satellite-based RS data are generally low, the phenomenon of "the same object with different spectrum" and "the different object with same spectrum" often occurs when observing grassland communities on a large scale [22,38], thus reducing the accuracy of classification.…”

“…The other reason for the difficulty in distinguishing grassland communities using spaceborne mid-resolution RS images is that the performance of the classification models used to classify grassland communities is suboptimal [3]. In previous studies of vegetation classification, classification models (including classifiers and their hyperparameters) were usually determined empirically without selecting the optimal ones based on the specific study [22,23]. Notably, studies suggest that the selection of classification models has a significant impact on the classification results [24,25].…”

Fish survival prediction in an aquatic environment using random forest model

“…Classification has used for prediction purposes; traditional rule-based algorithm does not provide any prediction feature for the unknown dataset. Confusion matrix provides various measurement of accuracy in prediction, where rule-based algorithm cannot perform this [7]. CNN is a deep learning model where computation complexity is higher than machine learning models.…”

Multimodality in Meta-Learning: A Comprehensive Survey

“…Existing evidence has shown that by combining prior knowledge and context, humans can gain experience on multiple prior tasks over bounded episodes, where the learned abstract experience is generalized to improve future learning performance on new concepts. Inspired by this, a computational paradigm called meta-learning [7,8] is proposed to simulate the ability of humans to learn generalized task experience, aiming to allow machines to acquire prior knowledge from similar tasks and quickly adapt to new tasks. The meta-learning process is more data-efficient than traditional machine learning models by extracting the crossdomain task goals in a dynamic selection [9,10].…”