Incremental Cost-Sensitive Support Vector Machine With Linear-Exponential Loss

Abstract

Incremental learning or online learning as a branch of machine learning has attracted more attention recently. For large-scale problems and dynamic data problem, incremental learning overwhelms batch learning, because of its efficient treatment for new data. However, class imbalance problem, which always appears in online classification brings a considerable challenge for incremental learning. The serious class imbalance problem may directly lead to a useless learning system. Cost-sensitive learning is an important learning paradigm for class imbalance problems and widely used in many applications. In this paper, we propose an incremental cost-sensitive learning method to tackle the class imbalance problems in the online situation. This proposed algorithm is based on a novel cost-sensitive support vector machine, which uses the Linear-exponential (LINEX) loss to implement high cost for minority class and low cost for majority class. Using the half-quadratic optimization, we first put forward the algorithm for the costsensitive support vector machine, called CSLINEX-SVM*. Then we propose the incremental cost-sensitive algorithm, ICSL-SVM. The results of numeric experiments demonstrate that the proposed incremental algorithm outperforms some conventional batch algorithms except the proposed CSLINEX-SVM*.

Asymmetric Loss-Oriented Cost Sensitive Learning Model for Credit Risk Assessment

“…Consider an augmented dataset {(x i , y i )} l i=1 ={(x i ;1),y i )} l i=1 ⊆ X × Y , where x i indicates the ith augmented representation by appending 1 to the original feature representation x i , for a simple form without the bias term; X and Y = {1, −1} are feature and label spaces respectively, l is the size of dataset. According to [3], the formulation of CSLINEX-SVM is provided as follows.…”

“…When dealing with imbalanced dataset, LINEX loss function can impose linear penalty on the misclassification of the majority class and exponential penalty on the misclassification of the minority class. More details can refer to [3]. Since CSLINEX-SVM can successfully handle imbalanced learning problem and credit risk assessment possesses class-imbalanced characteristic, we aim to adopted CSLINEX-SVM in the application of credit risk assessment.…”

“…Support vector machine (SVM) serves as a popular classification method with a solid theoretical foundation [2]. In [3], Ma et al proposed an asymmetric loss-oriented cost sensitive learning model named CSLINEX-SVM. The linear-exponential (LINEX) loss function is a smooth convex asymmetric function usually used in statistics.…”

Concept Drift Adaptation with Incremental–Decremental SVM

“…The algorithm was also adapted for regression [41][42][43]. The incremental approach was revisited more recently in [44], where a linear exponential cost-sensitive incremental SVM was defined. In the following, Equations ( 5)-( 14) are taken from [39].…”

A Machine Learning Method with Filter-Based Feature Selection for Improved Prediction of Chronic Kidney Disease

“…For example, the samples discarded from the majority class could be vital in efficiently training the classifiers [ 20 ]. Therefore, several studies have resorted to using algorithm-level methods such as ensemble learning and cost-sensitive learning to effectively handle the imbalanced data instead of data-level techniques [ 26 , 27 , 28 , 29 ].…”