Fast rates for support vector machines using Gaussian kernels

Abstract

For binary classification we establish learning rates up to the order of $n^{-1}$ for support vector machines (SVMs) with hinge loss and Gaussian RBF kernels. These rates are in terms of two assumptions on the considered distributions: Tsybakov's noise assumption to establish a small estimation error, and a new geometric noise condition which is used to bound the approximation error. Unlike previously proposed concepts for bounding the approximation error, the geometric noise assumption does not employ any smoothness assumption.Comment: Published at http://dx.doi.org/10.1214/009053606000001226 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Rademacher Chaos Complexities for Learning the Kernel Problem

“…From the standard error decomposition (Zhang, 2004;Bartlett et al, 2006;Steinwart and Scovel, 2007;Ying and Zhou, 2007), we have that…”

Finite-Dimensional Projection for Classification and Statistical Learning

“…• There is, up to our knowledge, no definite reference bound agreed upon that would accurately depict the behavior of the SVM, but there is a diversity of performance bounds to choose from in the recent literature (e.g., [11,17,18]). Here, we have chosen to compare the bound of Theorem 1 to the performance bound shown in [19].…”

“…It is more difficult to draw a meaningful comparison with other known bounds on the SVM; for example, in [17], a Gaussian kernel with width depending on the sample size n as well as on some "geometric" assumptions on P (Y |X) is considered. Here our focus was on a fixed kernel.…”

Sufficient Dimension Reduction via Squared-Loss Mutual Information Estimation

“…This quantity represents a complexity of function class G-the larger γ is, the more complex the function class G is. The Gaussian RKHS satisfies this condition for arbitrarily small γ (Steinwart & Scovel, 2007). Then we have the following theorem (its proof is omitted due to lack of space; we used Theorem 5.11 in van de Geer (2000)).…”