Sample Size for Comparing Correlated Concordance Rates

Abstract

We consider collecting the measurements of a gold standard and two methods with error from each site of subjects. We propose an asymptotic test statistic to compare the concordance rates of two methods with the gold standard and a closed-form sample size formula. Through simulations, we show that the test statistic accurately controls the type I error in small sample sizes, and the sample size formula accurately maintains the power in various settings. The proposed test statistic and the sample size formula can be easily modified for other indices for agreement such as sensitivity and specificity. A real eye study is taken as an example.

Design and Analysis Methods for Trials with AI-Based Diagnostic Devices for Breast Cancer

“…Existing papers on comparing correlated concordance rates mainly focus on comparing two (or more) competitive diagnosis methods using their concordance rates with a gold standard on multiple sites [11]. In this paper, there is no gold standard and we compare the concordance rate between an AI-based diagnostic device and human radiologists and that among radiologists.…”