An Efficient Method for Computing Leave-One-Out Error in Support Vector Machines With Gaussian Kernels

Lee, M.M.S.; Keerthi, S. Sathiya; Ong, Chong Jin; DeCoste, Dennis

doi:10.1109/tnn.2004.824266

Cited by 52 publications

(38 citation statements)

References 11 publications

(17 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…respectively i 1 and i 2 in equations (2d)). A recent alternative is to select the first α i using previous heuristic (i 1 for example) and to use equation (5) to select the second [7]. After selecting good candidates, the OptimalVariation procedure computes Δα i1 and Δα i2 values in order to have the maximal decrease of W (see [3,7] for more details).…”

Section: Svm and Smo Overviewmentioning

confidence: 99%

“…A better way to do this with SVMs is to first realize a training with all the m examples. This first optimal solution α * provides several useful informations [5]. Those informations allow to determine values of l(h Si θ (x i ), y i ) without any training with several datasets [5] for more details).…”

Section: Loo-cv Definitionmentioning

confidence: 99%

“…This first optimal solution α * provides several useful informations [5]. Those informations allow to determine values of l(h Si θ (x i ), y i ) without any training with several datasets [5] for more details). Let I LOO denote set of examples for which SVM trainings with datasets S i are necessary.…”

Section: Loo-cv Definitionmentioning

confidence: 99%

“…SVM is an efficient inducer, but training time increases quickly with training set size [3] and it would be a bad candidate for model selection with direct LOO-CV. But others properties of SVM made it a good candidate for smart LOO-CV [4,5]. Decoste et al [4] and others [2,5] have proposed new methods to speed-up exact (or very close) evaluation of LOO error with SVM.…”

Section: Introductionmentioning

confidence: 99%

“…But others properties of SVM made it a good candidate for smart LOO-CV [4,5]. Decoste et al [4] and others [2,5] have proposed new methods to speed-up exact (or very close) evaluation of LOO error with SVM. All these methods are based either on changing initialization, stopping criterion, or both of SMO algorithm.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Speed-Up LOO-CV with SVM Classifier

Lebrun¹,

Lézoray²,

Charrier³

et al. 2006

Intelligent Data Engineering and Automated Learning – IDEAL 2006

View full text Add to dashboard Cite

Abstract. Leave-one-out Cross Validation (LOO-CV) gives an almost unbiased estimate of the expected generalization error. But the LOO-CV classical procedure with Support Vector Machines (SVM) is very expensive and cannot be applied when training set has more that few hundred examples. We propose a new LOO-CV method which uses modified initialization of Sequential Minimal Optimization (SMO) algorithm for SVM to speed-up LOO-CV. Moreover, when SMO's stopping criterion is changed with our adaptive method, experimental results show that speed-up of LOO-CV is greatly increased while LOO error estimation is very close to exact LOO error estimation.

show abstract

Section: Svm and Smo Overviewmentioning

confidence: 99%

Section: Loo-cv Definitionmentioning

confidence: 99%

Section: Loo-cv Definitionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Speed-Up LOO-CV with SVM Classifier

Lebrun¹,

Lézoray²,

Charrier³

et al. 2006

Intelligent Data Engineering and Automated Learning – IDEAL 2006

View full text Add to dashboard Cite

show abstract

Model selection and error estimation without the agonizing pain

Oneto

2018

WIREs Data Min & Knowl

View full text Add to dashboard Cite

How can we select the best performing data-driven model? How can we rigorously estimate its generalization error? Statistical learning theory (SLT) answers these questions by deriving nonasymptotic bounds on the generalization error of a model or, in other words, by delivering upper bounding of the true error of the learned model based just on quantities computed on the available data. However, for a long time, SLT has been considered only as an abstract theoretical framework, useful for inspiring new learning approaches, but with limited applicability to practical problems. The purpose of this review is to give an intelligible overview of the problems of model selection (MS) and error estimation (EE), by focusing on the ideas behind the different SLT-based approaches and simplifying most of the technical aspects with the purpose of making them more accessible and usable in practice. We start by presenting the seminal works of the 80s until the most recent results, then discuss open problems and finally outline future directions of this field of research.

show abstract