Recent Advances on Support Vector Machines Research

Abstract

Support vector machines (SVMs), with their roots in Statistical Learning Theory (SLT) and optimization methods, have become powerful tools for problem solution in machine learning. SVMs reduce most machine learning problems to optimization problems and optimization lies at the heart of SVMs. Lots of SVM algorithms involve solving not only convex problems, such as linear programming, quadratic programming, second order cone programming, semi-definite programming, but also non-convex and more general optimization problems, such as integer programming, semi-infinite programming, bi-level programming and so on. The purpose of this paper is to understand SVM from the optimization point of view, review several representative optimization models in SVMs, their applications in economics, in order to promote the research interests in both optimization-based SVMs theory and economics applications. This paper starts with summarizing and explaining the nature of SVMs. It then proceeds to discuss optimization models for SVM following three major themes. First, least squares SVM, twin SVM, AUC Maximizing SVM, and fuzzy SVM are discussed for standard problems. Second, support vector ordinal machine, semisupervised SVM, Universum SVM, robust SVM, knowledge based SVM and multi-instance SVM are then presented for nonstandard problems. Third, we explore other important issues such as lp-norm SVM for feature selection, LOOSVM based on minimizing LOO error bound, probabilistic outputs for SVM, and rule extraction from SVM. At last, several applications of SVMs to financial forecasting, bankruptcy prediction, credit risk analysis are introduced.

Predictive planning with neural networks

“…With the real world problem presented in this paper solved there is also now scope for further work investigating the suitability of alternate techniques, such as support vector machines [19].…”

Semi-supervised classification with privileged information

“…More intuitively, smaller jf ðx i Þ À f ðx j Þj is, more smooth f(x) in the data adjacency graph is. So (17) can be translated to the following optimization…”

Twin support vector machine in linear programs

“…With the evolution of SVMs, they have shown much advantages in classification with small samples, nonlinear classification and high dimensional pattern recognition and also they can be applied in solving other machine learning problems [4][5][6][7][8][9][10]. The standard support vector classification attempts to minimize generalization error by maximizing the margin between two parallel hyperplanes, which results in dealing with an optimization task involving the minimization of a convex quadratic function.…”