Exploring Study Design Foibles in Randomized Controlled Trials on Convalescent Plasma in Hospitalized COVID-19 Patients

Abstract

Background: Sample size estimation is an essential step in the design of randomized controlled trials (RCTs) evaluating a treatment effect. Sample size is a critical variable in determining statistical significance and, thus, it significantly influences RCTs’ success or failure. During the COVID-19 pandemic, many RCTs tested the efficacy of COVID-19 convalescent plasma (CCP) in hospitalized patients but reported different efficacies, which could be attributed to, in addition to timing and dose, inadequate sample size estimates. Methods: To assess the sample size estimation in RCTs evaluating the effect of treatment with CCP in hospitalized COVID-19 patients, we searched the medical literature between January 2020 and March 2024 through PubMed and other electronic databases, extracting information on expected size effect, statistical power, significance level, and measured efficacy. Results: A total of 32 RCTs were identified. While power and significance level were highly consistent, heterogeneity in the expected size effect was relevant. Approximately one third of the RCTs did not reach the planned sample size for various reasons, with the most important one being slow patient recruitment during the pandemic’s peaks. RCTs with a primary outcome in favor of CCP treatment had a significant lower median absolute difference in the expected size effect than unfavorable RCTs (20.0% versus 33.9%, P = 0.04). Conclusions: The analyses of sample sizes in RCTs of CCP treatment in hospitalized COVID-19 patients reveal that many underestimated the number of participants needed because of excessively high expectations on efficacy, and thus, these studies had low statistical power. This, in combination with a lower-than-planned recruitment of cases and controls, could have further negatively influenced the primary outcomes of the RCTs.