Calibrated asymmetric surrogate losses

Scott, Clayton

doi:10.1214/12-ejs699

Cited by 67 publications

(76 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the problem that the 0-1 loss function is neither convex nor smooth, abundant convex surrogate loss functions (most are smooth) with the classificationcalibrated property [11], [12] have been proposed. These surrogate loss functions, such as square loss, logistic loss, and hinge loss, have been proven useful in many realworld applications.…”

Section: The Traditional Classification Pr-oblemmentioning

confidence: 99%

“…The method is notable because it can be applied to all the convex and classification-calibrated surrogate loss functions (If a surrogate loss function is classification-calibrated and the sample size is sufficiently large, the surrogate loss function will help learn the same optimal classifier as the 0-1 loss function does, see Theorem 1 in Bartlett et al [11]). This modification is based on the asymmetric classification-calibrated results [12] and cannot be used…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Classification with Noisy Labels by Importance Reweighting

Liu

Tao

2016

IEEE Trans. Pattern Anal. Mach. Intell.

665

403

View full text Add to dashboard Cite

Abstract-In this paper, we study a classification problem in which sample labels are randomly corrupted. In this scenario, there is an unobservable sample with noise-free labels. However, before being observed, the true labels are independently flipped with a probability ρ ∈ [0, 0.5), and the random label noise can be class-conditional. Here, we address two fundamental problems raised by this scenario. The first is how to best use the abundant surrogate loss functions designed for the traditional classification problem when there is label noise. We prove that any surrogate loss function can be used for classification with noisy labels by using importance reweighting, with consistency assurance that the label noise does not ultimately hinder the search for the optimal classifier of the noise-free sample. The other is the open problem of how to obtain the noise rate ρ. We show that the rate is upper bounded by the conditional probability P (Ŷ |X) of the noisy sample. Consequently, the rate can be estimated, because the upper bound can be easily reached in classification problems. Experimental results on synthetic and real datasets confirm the efficiency of our methods.

show abstract

Section: The Traditional Classification Pr-oblemmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Classification with Noisy Labels by Importance Reweighting

Liu

Tao

2016

IEEE Trans. Pattern Anal. Mach. Intell.

665

403

View full text Add to dashboard Cite

show abstract

“…In Fig.3(c), the positives lie farther from the ideal boundary than the negatives, thus we think that placing balanced margins may increase the bias of the model. Considering this issue and the viewpoint of asymmetric surrogate loss [15], we suggest the margin of positives should be larger in imbalanced classification so as to help improve the classification of positives. In all, we combine the ideas of DEC [19] and Margin Calibration [20], and introduce the asymmetric stagewise least square (ASLS) loss function in this paper:…”

Section: Asymmetric Stagewise Least Square Lossmentioning

confidence: 99%

“…Cost-sensitive learning [14] and asymmetric surrogate loss [15] are mainly used in internal methods. Support vector machines (SVMs) [16] are popular classifiers because of their remarkable generalization performance.…”

Section: Introductionmentioning

confidence: 99%

“…Thus, it is hopeful to extend the SLS loss to imbalanced cases so that its application is expanded. In this paper, we combine the ideas of DEC [19] and Margin Calibration [20] and propose an asymmetric stagewise least square (ASLS) loss function which is also a kind of asymmetric surrogate loss [15]. On one hand, we apply different penalties to different classes which is widely used in imbalanced classification.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An asymmetric stagewise least square loss function for imbalanced classification

Xu¹,

Hu²,

Prı́ncipe

2014

2014 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

In this paper, we present an asymmetric stagewise least square (ASLS) loss function for imbalanced classification. While keeping all the advantages of the stagewise least square (SLS) loss function, such as, better robustness, computational efficiency and sparseness, the ASLS loss extends the SLS loss by adding another two parameters, namely, ramp coefficient and margin coefficient. Therefore, asymmetric ramps and margins can be formed which makes the ASLS loss be more flexible and appropriate for processing class imbalance problems. A reduced kernel classifier of the ASLS loss is also developed which only uses a small part of the dataset to generate an efficient nonlinear classifier. Experimental results confirm the effectiveness of the ASLS loss in imbalanced classification.

show abstract

Cost Sensitive Learning in the Presence of Symmetric Label Noise

Tripathi

Hemachandra

2019

Advances in Knowledge Discovery and Data Mining

View full text Add to dashboard Cite

In binary classification framework, we are interested in making cost sensitive label predictions in the presence of uniform/symmetric label noise. We first observe that 0-1 Bayes classifiers are not (uniform) noise robust in cost sensitive setting. To circumvent this impossibility result, we present two schemes; unlike the existing methods, our schemes do not require noise rate. The first one uses α-weighted γ-uneven margin squared loss function, lα,usq, which can handle cost sensitivity arising due to domain requirement (using user given α) or class imbalance (by tuning γ) or both. However, we observe that lα,usq Bayes classifiers are also not cost sensitive and noise robust. We show that regularized ERM of this loss function over the class of linear classifiers yields a cost sensitive uniform noise robust classifier as a solution of a system of linear equations. We also provide a performance bound for this classifier. The second scheme that we propose is a re-sampling based scheme that exploits the special structure of the uniform noise models and uses in-class probability η estimates. Our computational experiments on some UCI datasets with class imbalance show that classifiers of our two schemes are on par with the existing methods and in fact better in some cases w.r.t. Accuracy and Arithmetic Mean, without using/tuning noise rate. We also consider other cost sensitive performance measures viz., F measure and Weighted Cost for evaluation. If the noise is adversarial then, one can interpret not requiring the noise rates as a dominant strategy and cross-validation over noise rates as a search for a dominant strategy. Also, the required restriction of the hypothesis class to the linear class can be interpreted as an implicit form of regularization. As our re-sampling scheme requires estimates of η, we provide a detailed comparative study of various η estimation methods on synthetic datasets, w.r.t. half a dozen evaluation criterion. Also, we provide understanding on the interpretation of cost parameters α and γ using different synthetic data experiments.

show abstract

Calibrated asymmetric surrogate losses

Cited by 67 publications

References 22 publications

Classification with Noisy Labels by Importance Reweighting

Classification with Noisy Labels by Importance Reweighting

An asymmetric stagewise least square loss function for imbalanced classification

Cost Sensitive Learning in the Presence of Symmetric Label Noise

Contact Info

Product

Resources

About