Support Vector Machines with the Ramp Loss and the Hard Margin Loss

Brooks, J. Paul

doi:10.1287/opre.1100.0854

Cited by 124 publications

(148 citation statements)

References 26 publications

Supporting

Mentioning

137

Contrasting

Unclassified

Order By: Relevance

“…Coordinate descent algorithm for ramp-LPSVM. ⋄ Express and solve the corresponding subproblem (6). Record the optimum z ⋆ k,j ;…”

Section: Coordinate Descent Algorithmmentioning

confidence: 99%

“…Recall the subproblem (6). Unlike a general nonlinear optimization problem, the one-variable piecewise linearity makes it convenient to be solved.…”

Section: Solving Subproblemsmentioning

confidence: 99%

“…In this way, the subproblem is transformed to a convex problem in R and the computational cost for an optimum is much less than that when solving problem (6). A useful property of the subgradient ofĥ is provided as follows, which will be exploited in the convergence analysis in the successive section.…”

Section: Solving Subproblemsmentioning

confidence: 99%

“…Wu and Liu present the theoretical analysis of the ramp loss SVM and investigate its advantages in sparsity and robustness in [33]. Later researchers reformulate SVM with ramp loss as mixed integer nonlinear programming problems, and then use a branch and bound method [19], heuristic algorithms [6] or commercial softwares [7] to obtain a solution. Another method proposed by Collobert et al in [9] is based on transforming the non-convex ramp loss SVM into a series of convex problems which can be solved by concave-convex procedure.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Coordinate Descent Algorithm for Ramp Loss Linear Programming Support Vector Machines

Huang

Suykens

et al. 2015

Neural Process Lett

View full text Add to dashboard Cite

In order to control the effects of outliers in training data and get sparse results, Huang et al. [15] proposed the ramp loss linear programming support vector machine (ramp-LPSVM). This combination of l 1 regularization and ramp loss does not only lead to the sparsity of parameters in decision functions, but also limits the effects of outliers with a maximal penalty. However, due to its non-convexity, the computational cost to achieve a satisfying solution is often expensive. In this paper, we propose a modified coordinate descent algorithm, which deals with a series of one-variable piecewise linear subproblems. Considering that the obtained subproblems are DC programming problems, we linearize the concave part of the objective functions and solve the obtained convex problems. To test the performances of the proposed algorithm, numerical experiments have been carried out and analysed on benchmark data sets. To enhance the sparsity and robustness, the experiments are initialized from C-SVM solutions. The results confirm its excellent performances in classification accuracy, robustness and efficiency in computation.

show abstract

“…Coordinate descent algorithm for ramp-LPSVM. ⋄ Express and solve the corresponding subproblem (6). Record the optimum z ⋆ k,j ;…”

Section: Coordinate Descent Algorithmmentioning

confidence: 99%

“…Recall the subproblem (6). Unlike a general nonlinear optimization problem, the one-variable piecewise linearity makes it convenient to be solved.…”

Section: Solving Subproblemsmentioning

confidence: 99%

Section: Solving Subproblemsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Coordinate Descent Algorithm for Ramp Loss Linear Programming Support Vector Machines

Huang

Suykens

et al. 2015

Neural Process Lett

View full text Add to dashboard Cite

show abstract

“…Thus, a classifier more robust against outliers is obtained, [241]. In [38], a nonlinear mixed-integer formulation of the SVM with ramp loss is proposed, where a binary variable is introduced for each training object equal to 1 if the object is misclassified outside the margin and 0 otherwise. The ramp loss has also been advocated in [213,66] with the motivation of generating less support vectors than the SVM with hinge loss.…”

Section: Departing From Convexity In Svmmentioning

confidence: 99%

Supervised classification and mathematical optimization

Carrizosa

Morales

2013

Computers & Operations Research

124

View full text Add to dashboard Cite

Data Mining techniques often ask for the resolution of optimization problems. Supervised Classification, and, in particular, Support Vector Machines, can be seen as a paradigmatic instance. In this paper, some links between Mathematical Optimization methods and Supervised Classification are emphasized. It is shown that many different areas of Mathematical Optimization play a central role in off-the-shelf Supervised Classification methods. Moreover, Mathematical Optimization turns out to be extremely useful to address important issues in Classification, such as identifying relevant variables, improving the interpretability of classifiers or dealing with vagueness/noise in the data.

show abstract

Breast tumor classification through learning from noisy labeled ultrasound images

et al. 2019

View full text Add to dashboard Cite

Purpose To train deep learning models to differentiate benign and malignant breast tumors in ultrasound images, we need to collect many training samples with clear labels. In general, biopsy results can be used as benign/malignant labels. However, most clinical samples generally do not have biopsy results. Previous works have proposed generating benign/malignant labels according to Breast Imaging, Reporting and Data System (BI‐RADS) ratings. However, this approach will cause noisy labels, which means that the benign/malignant labels produced from BI‐RADS diagnoses may be inconsistent with the ground truths. Consequently, deep models will overfit the noisy labels and hence obtain poor generalization performance. In this work, we mainly focus on how to reduce the negative effect of noisy labels when they are used to train breast tumor classification models. Methods We propose an effective approach called noise filter network (NF‐Net) to address the problem of noisy labels when training breast tumor classification models. Specifically, to prevent deep models from overfitting the noisy labels, we propose incorporating two softmax layers for classification. Additionally, to strengthen the effect of clean labels, we design a teacher–student module for distilling the knowledge of clean labels. Results We conduct extensive comparisons with the existing works on addressing noisy labels. Our method achieves a classification accuracy of 73%, with a precision of 69%, recall of 80%, and F1‐score of 0.74. This result is significantly better than those of the existing state‐of‐the‐art works on addressing noisy labels. Conclusions This work provides a means to overcome the label shortage problem in training breast tumor classification models. Specifically, we can generate benign/malignant labels according to the BI‐RADS ratings. Although this approach will cause noisy labels, the design of NF‐Net can effectively reduce the negative effect of such labels.

show abstract

Support Vector Machines with the Ramp Loss and the Hard Margin Loss

Cited by 124 publications

References 26 publications

Coordinate Descent Algorithm for Ramp Loss Linear Programming Support Vector Machines

Coordinate Descent Algorithm for Ramp Loss Linear Programming Support Vector Machines

Supervised classification and mathematical optimization

Breast tumor classification through learning from noisy labeled ultrasound images

Contact Info

Product

Resources

About