How to Compare Different Loss Functions and Their Risks

Steinwart, Ingo

doi:10.1007/s00365-006-0662-3

Cited by 111 publications

(220 citation statements)

References 28 publications

Supporting

Mentioning

213

Contrasting

Order By: Relevance

“…However, Theorem 3.16 of Steinwart [7] which is used in the proof of Theorem 6 actually provides a framework to replace · 0 by a more general notion of closedness if the assumption F * ,P (x) = { f * ,P (x)} is violated. In some sense the convergence with respect to · 0 is rather weak and one may wonder whether it can be replaced by some stronger notion of convergence.…”

Section: Theoremmentioning

confidence: 99%

Consistency of kernel‐based quantile regression

Christmann

Steinwart

2008

Appl Stoch Models Bus & Ind

Self Cite

View full text Add to dashboard Cite

SUMMARYQuantile regression is used in many areas of applied research and business. Examples are actuarial, financial or biometrical applications. We show that a non-parametric generalization of quantile regression based on kernels shares with support vector machines the property of consistency to the Bayes risk. We further use this consistency to prove that the non-parametric generalization approximates the conditional quantile function which gives the mathematical justification for kernel-based quantile regression.

show abstract

Section: Theoremmentioning

confidence: 99%

Consistency of kernel‐based quantile regression

Christmann

Steinwart

2008

Appl Stoch Models Bus & Ind

Self Cite

View full text Add to dashboard Cite

show abstract

“…This is known as the problem of calibration, which studies how small the suboptimality gap, measured in term of the surrogate risk, should be to achieve a suboptimality gap, measured in term of the risk of interest, of a given size. This has been studied in the binary cost-insensitive case (Bartlett et al 2006), in the binary cost-sensitive case (Steinwart 2007) and in the costsensitive multi-classe case (Pires et al 2013), for the convex surrogate proposed in Lee et al (2004). However, it has also been advocated recently that a surrogate loss function should be guess-averse (Beijbom et al 2014), in the sense that the loss should encourage more correct classifications than arbitrary guesses.…”

Section: Motivationmentioning

confidence: 99%

Soft-max boosting

Geist

2015

Mach Learn

View full text Add to dashboard Cite

The standard multi-class classification risk, based on the binary loss, is rarely directly minimized. This is due to (1) the lack of convexity and (2) the lack of smoothness (and even continuity). The classic approach consists in minimizing instead a convex surrogate. In this paper, we propose to replace the usually considered deterministic decision rule by a stochastic one, which allows obtaining a smooth risk (generalizing the expected binary loss, and more generally the cost-sensitive loss). Practically, this (empirical) risk is minimized by performing a gradient descent in the function space linearly spanned by a base learner (a.k.a. boosting). We provide a convergence analysis of the resulting algorithm and experiment it on a bunch of synthetic and real-world data sets (with noiseless and noisy domains, compared to convex and non-convex boosters).

show abstract

“…Clearly, minimizing the standard hinge loss is a special case of minimizing L(y, f, α). Steinwart (2007) studied theoretical properties of (5). The SVM with (5) was discussed in Lin et al (2002).…”

Section: Loss Functionsmentioning

confidence: 99%

HingeBoost: ROC-Based Boost for Classification and Variable Selection

Wang

2011

The International Journal of Biostatistics

View full text Add to dashboard Cite

In disease classification, a traditional technique is the receiver operative characteristic (ROC) curve and the area under the curve (AUC). With high-dimensional data, the ROC techniques are needed to conduct classification and variable selection. The current ROC methods do not explicitly incorporate unequal misclassification costs or do not have a theoretical grounding for optimizing the AUC. Empirical studies in the literature have demonstrated that optimizing the hinge loss can maximize the AUC approximately. In theory, minimizing the hinge rank loss is equivalent to minimizing the AUC in the asymptotic limit. In this article, we propose a novel nonparametric method HingeBoost to optimize a weighted hinge loss incorporating misclassification costs. HingeBoost can be used to construct linear and nonlinear classifiers. The estimation and variable selection for the hinge loss are addressed by a new boosting algorithm. Furthermore, the proposed twin HingeBoost can select more sparse predictors. Some properties of HingeBoost are studied as well. To compare HingeBoost with existing classification methods, we present empirical study results using data from simulations and a prostate cancer study with mass spectrometry-based proteomics.

show abstract

How to Compare Different Loss Functions and Their Risks

Cited by 111 publications

References 28 publications

Consistency of kernel‐based quantile regression

Consistency of kernel‐based quantile regression

Soft-max boosting

HingeBoost: ROC-Based Boost for Classification and Variable Selection

Contact Info

Product

Resources

About