The connection between regularization operators and support vector kernels

Smola, Alex; Schölkopf, Bernhard; Müller, Klaus‐Robert

doi:10.1016/s0893-6080(98)00032-x

Cited by 564 publications

(340 citation statements)

References 27 publications

(28 reference statements)

Supporting

Mentioning

328

Contrasting

Unclassified

Order By: Relevance

“…Recently, Support Vector Machines (SVMs) have been introduced by Vapnik (Boser, Guyon, & Vapnik, 1992;Vapnik, 1995) for solving classification and nonlinear function estimation problems (Cristianini & Shawe-Taylor, 2000;Schölkopf et al, 1997;Schölkopf, Burges, & Smola, 1998;Smola, Schölkopf, & Müller, 1998;Smola, 1999;Suykens & Vandewalle, 1998;Vapnik, 1995Vapnik, , 1998aVapnik, , 1998b. Within this new approach the training problem is reformulated and represented in such a way so as to obtain a (convex) quadratic programming (QP) problem.…”

Section: Introductionmentioning

confidence: 99%

Benchmarking Least Squares Support Vector Machine Classifiers

et al. 2004

View full text Add to dashboard Cite

Abstract. In Support Vector Machines (SVMs), the solution of the classification problem is characterized by a (convex) quadratic programming (QP) problem. In a modified version of SVMs, called Least Squares SVM classifiers (LS-SVMs), a least squares cost function is proposed so as to obtain a linear set of equations in the dual space. While the SVM classifier has a large margin interpretation, the LS-SVM formulation is related in this paper to a ridge regression approach for classification with binary targets and to Fisher's linear discriminant analysis in the feature space. Multiclass categorization problems are represented by a set of binary classifiers using different output coding schemes. While regularization is used to control the effective number of parameters of the LS-SVM classifier, the sparseness property of SVMs is lost due to the choice of the 2-norm. Sparseness can be imposed in a second stage by gradually pruning the support value spectrum and optimizing the hyperparameters during the sparse approximation procedure. In this paper, twenty public domain benchmark datasets are used to evaluate the test set performance of LS-SVM classifiers with linear, polynomial and radial basis function (RBF) kernels. Both the SVM and LS-SVM classifier with RBF kernel in combination with standard cross-validation procedures for hyperparameter selection achieve comparable test set performances. These SVM and LS-SVM performances are consistently very good when compared to a variety of methods described in the literature including decision tree based algorithms, statistical algorithms and instance based learning methods. We show on ten UCI datasets that the LS-SVM sparse approximation procedure can be successfully applied.

show abstract

Section: Introductionmentioning

confidence: 99%

Benchmarking Least Squares Support Vector Machine Classifiers

et al. 2004

View full text Add to dashboard Cite

show abstract

“…This is due to the fact that the capacity control performed by the SVM method is equivalent to some form of regularisation, so that "denoising" is not necessary [37]. In the case of KTA, the optimisation performed recognises directly the variables that do not report information about the labelling or that are very noisy.…”

Section: Filtering Non-informative Features For the Construction Of Tmentioning

confidence: 99%

A Study on Multi-Scale Kernel Optimisation via Centered Kernel-Target Alignment

Pérez-Ortiz

Gutiérrez

Sánchez‐Monedero

et al. 2015

Neural Process Lett

View full text Add to dashboard Cite

“…) Let the Mercer kernel defined by k : X × X → R, and the regularisation operator Γ : Smola, Schölkopf, and Müller, 1998. ) Consider a kernel, endowed with translation invariance, namely k(x, ξ) …”

Section: Model Constructionmentioning

confidence: 99%

A signal theory approach to support vector classification: The sinc kernel

et al. 2009

View full text Add to dashboard Cite

Fourier-based regularisation is considered for the support vector machine classification problem over absolutely integrable loss functions. By invoking the modest assumption that the decision function belongs to a Paley-Wiener space, it is shown that the classification problem can be developed in the context of signal theory. Furthermore, by employing the Paley-Wiener reproducing kernel, namely the sinc function, it is shown that a principled and finite kernel hyper-parameter search space can be discerned, a priori.Subsequent simulations performed on a commonly-available hyperspectral image data set reveal that the approach yields results that surpass state-of-the-art benchmarks.

show abstract

The connection between regularization operators and support vector kernels

Cited by 564 publications

References 27 publications

Benchmarking Least Squares Support Vector Machine Classifiers

Benchmarking Least Squares Support Vector Machine Classifiers

A Study on Multi-Scale Kernel Optimisation via Centered Kernel-Target Alignment

A signal theory approach to support vector classification: The sinc kernel

Contact Info

Product

Resources

About