A Meta-Learning Method to Select the Kernel Width in Support Vector Regression

Abstract

Abstract. The Support Vector Machine algorithm is sensitive to the choice of parameter settings. If these are not set correctly, the algorithm may have a substandard performance. Suggesting a good setting is thus an important problem. We propose a meta-learning methodology for this purpose and exploit information about the past performance of different settings. The methodology is applied to set the width of the Gaussian kernel. We carry out an extensive empirical evaluation, including comparisons with other methods (fixed default ranking; selection based on cross-validation and a heuristic method commonly used to set the width of the SVM kernel). We show that our methodology can select settings with low error while providing significant savings in time. Further work should be carried out to see how the methodology could be adapted to different parameter setting tasks.

Meta-learning to optimize the number of hidden nodes of MLP networks trained by Extreme Learning Machine algorithm

“…In this work, a set of meta-examples were built from 112 classification datasets. In [10], meta-learning was used to select the width parameter of the Gaussian kernel for regression problems. Although the width parameter of Gaussian kernel is continuous, the problem was treated as a classification task, with the meta-label variable assuming 11 different discrete values.…”

“…In our study we used 16 meta-attributes. The choice of meta-attributes was based on earlier meta-learning studies which work with regression problems [10] [9]:…”

Feature Selection Using Stochastic Search: An Application to System Identification

“…SVM hyper-parameters can be found through grid search (Soares et al 2004). Chapelle et al (2002) propose a gradient descent algorithm.…”

Introduction to the Special Issue on Meta-Learning

“…The first contribution to this issue (Soares, Brazdil, & Kuba, 2004) is a meta-learning approach to parameter setting (Section 2.1). The work focuses on Support Vector Machines and the goal is to set the width σ of the Gaussian kernel.…”