Marker selection via maximizing the partial area under the ROC curve of linear risk scores

Abstract

Rather than viewing receiver operating characteristic (ROC) curves directly to compare the performances of diagnostic methods, the whole and the partial areas under the ROC curve (area under the ROC curve [AUC] and partial area under the ROC curve [pAUC]) are 2 of the most popularly used summaries of the curve. Moreover, when high specificity is a prerequisite, as in some medical diagnostics, pAUC is preferable. In this paper, we propose a wrapper-type algorithm to select the best linear combination of markers that has high sensitivity within a confined specificity range. The markers selected by the proposed algorithm are different from those selected by AUC-based algorithms and therefore provide different information for further studies. Most notably, for example, within the given range of specificity, the markers selected by the proposed algorithm always have higher individual sensitivities than those selected by other AUC-based methods. This characteristic makes the proposed method a good addition to existing methods. Without assuming the underlying distributions of markers, we prove that the pAUC obtained with the proposed algorithm is a strongly consistent estimate of the true pAUC and then illustrate its performance with numerical studies using synthesized data and 2 real examples. The results are compared with those obtained by its AUC-based counterpart. We found that the classification performance of the final classifier based on the selected markers is very competitive.

A PAUC-based Estimation Technique for Disease Classification and Biomarker Selection

“…Although many authors have highlighted the importance of PAUC, there exist only few methods that use the PAUC as an objective function for finding optimal combinations of biomarkers (Dodd and Pepe 2003, Komori and Eguchi 2010, Wang and Chang 2010. To address this problem, we present a new technique for deriving marker combinations that is explicitly based on the PAUC criterion.…”

“…As outlined above, this problem is addressed by PAUC-GBS. Wang and Chang (2010) proposed a wrapper-type algorithm to optimize the PAUC over a combination of biomarkers. While the algorithm by Wang and Chang (2010) is applicable in both low-and high-dimensional settings, it is restricted to linear combinations of markers.…”

Boosting Learning Algorithm for Pattern Recognition and Beyond

“…In this context, the partial area under the ROC curve is getting more useful than the AUC itself. The classification problems relating to the pAUC are discussed in several papers such as [14]- [16].…”

On Feature Ensemble Optimizing the Sensitivity and Partial ROC Curve

“…Furthermore, Komori and Eguchi [2010], Ma and Huang [2007a] and Ye et al [2007] combined the ROC-based ensemble and various regularization approaches to construct the scoring system from a ultra-large number of features. Ricamato and Tortorella [2011] and Wang and Chang [2011] proposed similar approaches for optimizing area under the partial ROC curves. All the aforementioned methods are modelfree in the sense that the targeting function to be maximized approximates the area under ROC or partial ROC curves without the need of any parametric model assumption.…”

“…Assuming that the features of case and control follow distinct multivariate Gaussian distributions, Su and Liu [1993], Hsu and Hsueh [2013] and Hsu et al [2014] proposed to maximize the model-based estimate for the area under ROC or partial ROC curve. However, one remaining big obstacle is that the target function associated with the area under partial ROC curve including those proposed by Hsu and Hsueh [2013], Hsu et al [2014], Ricamato and Tortorella [2011] and Wang and Chang [2011], is ill-behaved with multiple local maximizers and there is no reliable numerical algorithm to find the global optimum. Furthermore, the asymptotical properties of the estimated combinations are difficult to study.…”