Confidence intervals of the difference between areas under two ROC curves in matched‐pair experiments

Abstract

In a matched-pair study, when outcomes of two diagnostic tests are ordinal/continuous, the difference between two correlated areas under ROC curves (AUCs) is usually used to compare the overall discriminatory ability of two diagnostic tests. This article considers confidence interval (CI) construction problems of difference between two correlated AUCs in a matched-pair experiment, and proposes 13 hybrid CIs based on variance estimates recovery with the maximum likelihood estimation, Delong's statistic, Wilson score statistic (WS) and WS with continuity correction, the modified Wald statistic (MW) and MW with continuity correction and Agresti-Coull statistic, and three Bootstrap-resampling-based CIs. For comparison, we present traditional parametric and nonparametric CIs. Simulation studies are conducted to assess the performance of the proposed CIs in terms of empirical coverage probabilities, empirical interval widths, and ratios of the mesial noncoverage probabilities to the noncoverage probabilities. Two examples from clinical studies are illustrated by the proposed methodologies. Empirical results evidence that the hybrid Agresti-Coull CI with the empirical estimation (EAC) behaved most satisfactorily because its coverage probability was quite close to the prespecified confidence level with short interval width. Hence, we recommend the usage of the EAC CI in applications.

Pulmonary transit time of cardiovascular magnetic resonance perfusion scans for quantification of cardiopulmonary haemodynamics

“…Confidence intervals as well as sensitivity and specificity were estimated using bootstrapping. 19 , 20 All statistical analyses were performed using Python 3.8.8.…”

Revision of the AAST grading scale for acute cholecystitis with comparison to physiologic measures of severity

“…For each of these models, discriminatory power was assessed by generating ROCs and calculating AUC for each of these models. The AUCs were then compared using the methods of Delong 13 . All statistical analyses were performed with R: a language and environment for statistical computing v. 4.0.0 (R Foundation for Statistical Computing; Vienna, Austria) installed packages included: lmtest, pROC, ggplot2, arsenal, DescTools, dplyr.…”

“…The AUCs were then compared using the methods of Delong. 13 All statistical analyses were performed with R: a language and environment for statistical computing v. 4.0.0 (R Foundation for Statistical Computing; Vienna, Austria) installed packages included: lmtest, pROC, ggplot2, arsenal, DescTools, dplyr. The Yale School of Medicine Institutional Review Board and all local institutional review boards approved this study.…”