The Multiple Pairs SMO: A modified SMO algorithm for the acceleration of the SVM training

Hernandez, Raul Acosta; Strum, Marius; Chau, Wang Jiang; Gonzalez, Jose Artur Quilici

doi:10.1109/ijcnn.2009.5178701

Cited by 6 publications

(2 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To improve convergence, more than two variables are optimized at a time [10], [11], [12]. In [12] SMO-NM was proposed, in which SMO and Newton's method are fused.…”

Section: Introductionmentioning

confidence: 99%

Optimizing working sets for training support vector regressors by Newton's method

Abe

2015

2015 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

In this paper, we train support vector regressors (SVRs) fusing sequential minimal optimization (SMO) and Newton's method. We use the SVR formulation that includes the absolute variables. A partial derivative of the absolute variable with respect to the associated variable is indefinite when the variable takes on zero. We determine the derivative value according to whether the optimal solution exits in the positive region, negative region, or at zero. In selecting working set, we use the method that we have developed for the SVM, namely, in addition to the pair of variables selected by SMO, loop variables that repeatedly appear in training, are added to the working set. By this method the working set size is automatically determined. We demonstrate the validity of our method over SMO using several benchmark data sets.

show abstract

“…To improve convergence, more than two variables are optimized at a time [10], [11], [12]. In [12] SMO-NM was proposed, in which SMO and Newton's method are fused.…”

Section: Introductionmentioning

confidence: 99%

Optimizing working sets for training support vector regressors by Newton's method

Abe

2015

2015 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

show abstract

“…To improve convergence, more than two variables are optimized at a time [22][23][24][25][26][27]. In [22], q (≥ 2) modifiable variables are selected in the steepest ascent direction and are optimized.…”

mentioning

confidence: 99%

Fusing sequential minimal optimization and Newton’s method for support vector training

Abe

2014

Int. J. Mach. Learn. & Cyber.

View full text Add to dashboard Cite

Sequential Minimal Optimization (SMO) is widely used for training support vector machines (SVMs) because of fast training. But the training slows down when a large margin parameter value is used. Training by Newton's method (NM) accelerates training in such a situation but it slows down for a small margin parameter value. To solve this problem, in this paper we fuse SMO with NM and call it SMO-NM. Because slow training is caused by repetitive corrections of the same variables, we modify the working set selection when they are detected. We call the variables that are selected by SMO, SMO variables. At the current step, if a variable selected as an SMO variable was selected in a previous step, we consider that a loop is detected. And in addition to the SMO variables, we add, to the working set, the unbounded variables that were selected as SMO variables and correct the variables by NM. If no loop is detected, the training procedure is the same as that of SMO. As a variant of this working set strategy, we further add violating variables to the working set.We clarify that if the classification problem is not linearly separable in the feature space, the solutions of L1/L2 SVMs (with the linear sum/square sum of slack variables) are unbounded as the margin parameter value approaches infinity but that, if the mapped training data are not linearly independent in the feature space, the solution of the LS (Least Squares) SVM is unbounded as the margin parameter approaches infinity. We also clarify the condition, in which the increment of the objective function value by SMO-NM is larger than that by SMO.We evaluate SMO-NM for several benchmark data sets and confirm the effectiveness over SMO especially for a large margin parameter value.

show abstract

Training Methods

Abe¹

2010

Support Vector Machines for Pattern Classification

View full text Add to dashboard Cite

The Multiple Pairs SMO: A modified SMO algorithm for the acceleration of the SVM training

Cited by 6 publications

References 10 publications

Optimizing working sets for training support vector regressors by Newton's method

Optimizing working sets for training support vector regressors by Newton's method

Fusing sequential minimal optimization and Newton’s method for support vector training

Training Methods

Contact Info

Product

Resources

About