Efficient statistical tests to compare Youden index: accounting for contingency correlation

Abstract

Youden index is widely utilized in studies evaluating accuracy of diagnostic tests and performance of predictive, prognostic, or risk models. However, both one and two independent sample tests on Youden index have been derived ignoring the dependence (association) between sensitivity and specificity, resulting in potentially misleading findings. Besides, paired sample test on Youden index is currently unavailable. This article develops efficient statistical inference procedures for one sample, independent, and paired sample tests on Youden index by accounting for contingency correlation, namely associations between sensitivity and specificity and paired samples typically represented in contingency tables. For one and two independent sample tests, the variances are estimated by Delta method, and the statistical inference is based on the central limit theory, which are then verified by bootstrap estimates. For paired samples test, we show that the estimated covariance of the two sensitivities and specificities can be represented as a function of kappa statistic so the test can be readily carried out. We then show the remarkable accuracy of the estimated variance using a constrained optimization approach. Simulation is performed to evaluate the statistical properties of the derived tests. The proposed approaches yield more stable type I errors at the nominal level and substantially higher power (efficiency) than does the original Youden's approach. Therefore, the simple explicit large sample solution performs very well. Because we can readily implement the asymptotic and exact bootstrap computation with common software like R, the method is broadly applicable to the evaluation of diagnostic tests and model performance.

Response to comment on ‘Efficient statistical tests to compare Youden index: accounting for contingency correlation’

“…As we pointed out in our article , this formula (7) is also supported by the results of simulation study and bootstrap exact computation . In the discussion , we emphasized that that we could use bootstrap to perform an exact test without using the approximation because bootstrap procedure can be readily implemented with existing software such as R. On the association between sensitivity and specificity. …”

“…We thank Drs. Reiser and Nakas for their comments about the variance formula in Cleophas that our article cited and utilized. Its mathematical derivation is more involved than a direct application of the Delta method.…”

Development of the Sydney Falls Risk Screening Tool in brain injury rehabilitation: A multisite prospective cohort study

“… Weighted SFRST and STRATIFY Youden indices compared accounting for contingency correlation (Chen et al., ). …”

“…McNemar's test and DeLong test were used to determine whether the developed model had greater clinical validity than STRATIFY [cut‐off score ≥9 (Papaioannou et al., )] (Kim & Lee, ). Youden indices were also compared accounting for contingency correlation (Chen, Xue, Tan, & Chen, ).…”

On ‘Efficient statistical tests to compare Youden index: accounting for contingency correlation’

“…Although Cleophas et al write this as the variance of the sum, it is clear from the context that this is a misprint and that they are in fact dealing with their product. Consequently, Equation (6) of Chen et al for the covariance of and is simply wrong, and all the statistical tests and discussions in this paper based on Equation (6) are useless.…”