Joint hypothesis testing of the area under the receiver operating characteristic curve and the Youden index

Abstract

In the receiver operating characteristic (ROC) analysis, the area under the ROC curve (AUC) serves as an overall measure of diagnostic accuracy. Another popular ROC index is the Youden index (J), which corresponds to the maximum sum of sensitivity and specificity minus one. Since the AUC and J describe different aspects of diagnostic performance, we propose to test if a biomarker beats the pre-specified targeting values of AUC 0 and J 0 simultaneously with H 0 : AUC ≤ AUC 0 or J ≤ J 0 against H a : AUC > AUC 0 and J > J 0 . This is a multivariate order restrictive hypothesis with a non-convex space in H a , and traditional likelihood ratio-based tests cannot apply. The intersection-union test (IUT) and the joint test are proposed for such test. While the IUT test independently tests for the AUC and the Youden index, the joint test is constructed based on the joint confidence region. Findings from the simulation suggest both tests yield similar power estimates. We also illustrated the tests using a real data example and the results of both tests are consistent. In conclusion, testing jointly on AUC and J gives more reliable results than using a single index, and the IUT is easy to apply and have similar power as the joint test.

Adrenomedullin Is a Diagnostic and Prognostic Biomarker for Acute Intracerebral Hemorrhage

“…Bivariate analyses were performed with Pearson's χ2 test, modified by Fisher's exact test, or in the case of continuous variables with non-parametric tests such as Kruskal-Wallis followed by Mann-Whitney's U test. The optimal AM cut-off point to differentiate between stroke patients and healthy controls was determined by Youden's J statistic [36].…”

Familial Inference

“…Berger and Hsu (1996) generalize these conditions and also provide an example where an initially conservative IU test can be modified to improve its frequentist power characteristics. Li, Cao, and Zhang (2020) and Yin, Mutiso, and Tian (2021) contain some recent applications.…”

In reference to intraoperative dexmedetomidine for prevention of postoperative delirium in elderly patients with mild cognitive impairment

“…2021;36:1592-1593. wileyonlinelibrary.com/journal/gps © 2021 John Wiley & Sons Ltd. positive predictive ability. 9 After the ROC analysis, moreover, the authors only provided the specificity and sensitivity of the multivariate model, but not its positive and negative predictive values Thus, we cannot determine whether their multivariate model really is useful for prediction of POD in clinical practice.…”