On support vector machines under a multiple-cost scenario

Abstract

Support vector machine (SVM) is a powerful tool in binary classification, known to attain excellent misclassification rates. On the other hand, many realworld classification problems, such as those found in medical diagnosis, churn or fraud prediction, involve misclassification costs which may be different in the different classes. However, it may be hard for the user to provide precise values for such misclassification costs, whereas it may be much easier to identify acceptable misclassification rates values. In this paper we propose a novel SVM model in which misclassification costs are considered by incorporating performance constraints in the problem formulation. Specifically, our aim is to seek the hyperplane with maximal margin yielding misclassification rates below given threshold values. Such maximal margin hyperplane is obtained by solving a quadratic convex problem with linear constraints and integer variables. The reported numerical experience shows that our model gives the user control on the misclassification rates in one class (possibly at the expense of an increase in misclassification rates for the other class) and is feasible in terms of running times.

Cost-sensitive Feature Selection for Support Vector Machines

“…In fact, experiments are done so that new performance measurements will not be 0.025 points lower than the originals (those obtained under the standard version of the SVM with linear kernel). Using the notation as in [31] (where 10) and (11).…”

“…For z (and thus K z ) fixed, the aim is to solve (4), but replacing the terms w w and w φ(x i ), respectively, by the expressions ( 6) and ( 7), apart from adding the constraints related to the performance measurements, as described in [31]. Therefore, the cost-sensitive sparse SVM with an arbitrary kernel K is defined (once z is fixed) as…”

“…The core idea is the optimization problem to be solved: instead of maximizing the margin, as in the traditional SVM, we seek the classifier with lowest number of features, but without damaging too much the original performance. In order to be able to control the classifier's performance, we will make use of constraints as in [31]. Specifically, the formulation of the constrained SVM with linear kernel is…”

“…In essence, this is simply the formulation for the SVM with linear kernel, to which performance constraints (µ(ζ) ≥ λ ) have been added, see [31] for the details. Its (partial) dual formulation is…”

On sparse ensemble methods: An application to short-term predictions of the evolution of COVID-19

“…In addition, our approach can easily incorporate cost-sensitive performance constraints to ensure that we control not only the overall accuracy of the regressor, but also the accuracy on a number of critical groups, as in Benítez-Peña et al. (2019a) , Benítez-Peña, Blanquero, Carrizosa, and Ramírez-Cobo (2019b) , Blanquero et al. (2020) and Datta and Das (2015) .…”