Support Vector Algorithms for Optimizing the Partial Area under the ROC Curve

Abstract

The area under the ROC curve (AUC) is a widely used performance measure in machine learning. Increasingly, however, in several applications, ranging from ranking to biometric screening to medicine, performance is measured not in terms of the full area under the ROC curve, but in terms of the partial area under the ROC curve between two false positive rates. In this paper, we develop support vector algorithms for directly optimizing the partial AUC between any two false positive rates. Our methods are based on minimizing a suitable proxy or surrogate objective for the partial AUC error. In the case of the full AUC, one can readily construct and optimize convex surrogates by expressing the performance measure as a summation of pairwise terms. The partial AUC, on the other hand, does not admit such a simple decomposable structure, making it more challenging to design and optimize (tight) convex surrogates for this measure.Our approach builds on the structural SVM framework of Joachims (2005) to design convex surrogates for partial AUC, and solves the resulting optimization problem using a cutting plane solver. Unlike the full AUC, where the combinatorial optimization needed in each iteration of the cutting plane solver can be decomposed and solved efficiently, the corresponding problem for the partial AUC is harder to decompose. One of our main contributions is a polynomial time algorithm for solving the combinatorial optimization problem associated with partial AUC. We also develop an approach for optimizing a tighter non-convex hinge loss based surrogate for the partial AUC using difference-of-convex programming. Our experiments on a variety of real-world and benchmark tasks confirm the efficacy of the proposed methods.

Epilepsy biomarkers – Toward etiology and pathology specificity

“…Standardized pAUC values can be used and compared like normal AUC values (Walter, 2005). Furthermore, Support Vector Machine-based estimation algorithms have been developed to optimize pAUC performance (Narasimhan and Agarwal, 2017). Better tools are needed to estimate biologically relevant sensitivity and specificity cut-off values for calculating the pAUC.…”

FCDP: Fidelity Calculation for Description-to-Permissions in Android Apps

“…For instance, CALENDAR permission is introduced in 287 sentences (1.16%). Since accuracy cannot reflect the prediction performance for imbalanced dataset [45], we selected two other metrics: Area Under Curve of Receiver Operating Characteristic curve (ROC-AUC) [46], and Area Under Curve of Precision-Recall Curve (PR-AUC) [47].…”

“…The ROC curve is a chart that visualizes the trade-off between the true positive rate (TPR) and false-positive rate (FPR) on the different threshold [46]. ROC-AUC is the area under the ROC curve.…”

A stream-sensitive distributed approach for configuring cascaded classifier topologies in real-time large-scale stream mining systems

“…If we assume that each classifier operates at a fixed complexity level, i and i can become deterministic function of P F i by using the APP i , and the detection error trade-off (DET) curve of the classifier, which is concave and increasing and relates P D i to P F i ( P D i = f P F i ) [30,38,39]. Therefore,…”