On Extending the SMO Algorithm Sub-Problem

Sentelle, Christopher; Georgiopoulos, Michael; Anagnostopoulos, Georgios C.; Young, Chieh-Neng

doi:10.1109/ijcnn.2007.4371075

Cited by 6 publications

(3 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Training methods using variable size chunking are sometimes called active set training. The well known training method with fixed-size chunking is Se quential Minimum Optimization (SMO) [5], which optimizes two data at a time and it is extended to optimizing four data at a time [6]. As a variant of decomposition-based training methods, the exact incremental training [7] is extended to batch training [8].…”

Section: Introductionmentioning

confidence: 99%

Convergence improvement of active set support vector training

Yabuwaki

Abe

2010

The 2010 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

Ahstract-In our previous work we have developed an active set training method of L2 support vector machines (SVMs) using Newton's method. Because the method allows a solution to be infeasible during training, convergence of the method is not guaranteed. In this paper, we guarantee convergence of active set training by limiting the corrections under the constraints when slow convergence is detected. Namely, we start training the L2 SVM with a subset of training data, delete non-positive dual variables from the working set as well as the variables with margins larger than or equal to 1, add violating variables to the working set, and repeat training. We monitor the number of violation fluctuations and if it exceeds a specified value, we obtain a feasible solution prohibiting addition of violating variables. Then for a feasible solution, we start active set training limiting the corrections within a feasible solution.By computer experiments, we show that the proposed training method is faster and more stable than the previous method.

show abstract

Section: Introductionmentioning

confidence: 99%

Convergence improvement of active set support vector training

Yabuwaki

Abe

2010

The 2010 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

show abstract

“…Kecman, Vogt, and Huang [5] showed that this algorithm is equivalent to the kernel-Adatron algorithm. Sentelle et al [6] extended SMO to optimizing four data at a time.…”

Section: Introductionmentioning

confidence: 99%

Artificial Neural Networks - ICANN 2008

Kůrková¹,

Neruda²,

Koutník³

2008

Lecture Notes in Computer Science

View full text Add to dashboard Cite

In this paper we discuss training support vector machines (SVMs) repetitively solving a set of linear equations, which is an extension of accurate incremental training proposed by Cauwenberghs and Poggio. First, we select two data in different classes and determine the optimal separating hyperplane. Then, we divide the training data set into several chunk data sets and set the two data to the active set, which includes current and previous support vectors. For the combined set of the active set and a chunk data set, we detect the datum that maximally violates the Karush-Kuhn-Tacker (KKT) conditions, and modify the optimal hyperplane by solving a set of equations that constrain the margins of unbounded support vectors and the optimality of the bias term. We iterate this procedure until there are no violating data in any combined set. By the computer experiment using several benchmark data sets, we show that the training speed of the proposed method is comparable with the primal-dual interior-point method combined with the decomposition technique, usually with a smaller number of support vectors.

show abstract

“…To improve convergence, more than two variables are optimized at a time [22][23][24][25][26][27]. In [22], q (≥ 2) modifiable variables are selected in the steepest ascent direction and are optimized.…”

mentioning

confidence: 99%

Fusing sequential minimal optimization and Newton’s method for support vector training

Abe

2014

Int. J. Mach. Learn. & Cyber.

View full text Add to dashboard Cite

Sequential Minimal Optimization (SMO) is widely used for training support vector machines (SVMs) because of fast training. But the training slows down when a large margin parameter value is used. Training by Newton's method (NM) accelerates training in such a situation but it slows down for a small margin parameter value. To solve this problem, in this paper we fuse SMO with NM and call it SMO-NM. Because slow training is caused by repetitive corrections of the same variables, we modify the working set selection when they are detected. We call the variables that are selected by SMO, SMO variables. At the current step, if a variable selected as an SMO variable was selected in a previous step, we consider that a loop is detected. And in addition to the SMO variables, we add, to the working set, the unbounded variables that were selected as SMO variables and correct the variables by NM. If no loop is detected, the training procedure is the same as that of SMO. As a variant of this working set strategy, we further add violating variables to the working set.We clarify that if the classification problem is not linearly separable in the feature space, the solutions of L1/L2 SVMs (with the linear sum/square sum of slack variables) are unbounded as the margin parameter value approaches infinity but that, if the mapped training data are not linearly independent in the feature space, the solution of the LS (Least Squares) SVM is unbounded as the margin parameter approaches infinity. We also clarify the condition, in which the increment of the objective function value by SMO-NM is larger than that by SMO.We evaluate SMO-NM for several benchmark data sets and confirm the effectiveness over SMO especially for a large margin parameter value.

show abstract

On Extending the SMO Algorithm Sub-Problem

Cited by 6 publications

References 14 publications

Convergence improvement of active set support vector training

Convergence improvement of active set support vector training

Artificial Neural Networks - ICANN 2008

Fusing sequential minimal optimization and Newton’s method for support vector training

Contact Info

Product

Resources

About