Construction of confidence regions in the ROC space after the estimation of the optimal Youden index‐based cut‐off point

Abstract

After establishing the utility of a continuous diagnostic marker investigators will typically address the question of determining a cut-off point which will be used for diagnostic purposes in clinical decision making. The most commonly used optimality criterion for cut-off point selection in the context of ROC curve analysis is the maximum of the Youden index. The pair of sensitivity and specificity proportions that correspond to the Youden index-based cut-off point characterize the performance of the diagnostic marker. Confidence intervals for sensitivity and specificity are routinely estimated based on the assumption that sensitivity and specificity are independent binomial proportions as they arise from the independent populations of diseased and healthy subjects, respectively. The Youden index-based cut-off point is estimated from the data and as such the resulting sensitivity and specificity proportions are in fact correlated. This correlation needs to be taken into account in order to calculate confidence intervals that result in the anticipated coverage. In this article we study parametric and non-parametric approaches for the construction of confidence intervals for the pair of sensitivity and specificity proportions that correspond to the Youden index-based optimal cut-off point. These approaches result in the anticipated coverage under different scenarios for the distributions of the healthy and diseased subjects. We find that a parametric approach based on a Box-Cox transformation to normality often works well. For biomarkers following more complex distributions a non-parametric procedure using logspline density estimation can be used.

Quality Control of Serum and Plasma by Quantification of (4E,14Z)-Sphingadienine-C18-1-Phosphate Uncovers Common Preanalytical Errors During Handling of Whole Blood

“…These analyses were also used to calculate the analytical sensitivity and specificity. The Youden index was then used in accordance with our previous study (21 ) to identify an optimal cutoff value to characterize good-quality plasma. P values Ͻ0.05 and a false discovery rate of Ͻ0.05 were considered statistically significant.…”

Construction of confidence intervals for the maximum of the Youden index and the corresponding cutoff point of a continuous biomarker

“…This property of the Box-Cox approach has been noted in other articles of the ROC-related literature (Bantis et al, 2014(Bantis et al, , 2017Molodianovitch et al, 2006;Schisterman et al, 2006). This approach has the advantages of parametric methodology and also has the flexibility of non-parametric methods in that it is applicable to a wide variety of distributional scenarios.…”

“…This has been pointed out before in the literature by Bantis et al (2014). For the parametric approach we focused on the often used binormal model that assumes that a common monotonic transformation to normality exists for both the healthy and diseased groups.…”

“…There are two possible ways to proceed. Recently, in a related problem, Bantis, Nakas, and Reiser (2014) found that in order to obtain satisfactory coverage of confidence intervals the variability of the estimated transformation parameter needs to be taken into account. We usê( ) to denote the estimate of obtained as in Section 2.1, but using the transformed observations.…”

“…Bantis et al (2014) describe in detail howΣ ( ) , an estimator of the covariance matrix of the estimated parameter vector Bantis et al (2014) describe in detail howΣ ( ) , an estimator of the covariance matrix of the estimated parameter vector…”