Sample size calculation for studies comparing binary outcomes using historical controls

Abstract

In historical control trials (HCTs), the experimental therapy is compared with a control therapy that has been evaluated in a previously conducted trial. Makuch and Simon developed a sample size formula where the observations from the HC group were considered not subject to sampling variability. Many researchers have pointed out that the Makuch-Simon sample size formula does not preserve the nominal power and type I error. We develop a sample size calculation approach that properly accounts for the uncertainty in the true response rate of the HC group. We demonstrate that the empirical power and type I error, obtained over the simulated HC data, have extremely skewed distributions. We then derive a closed-form sample size formula that enables researchers to control percentiles, instead of means, of the power and type I error accounting for the skewness of the distributions. A simulation study demonstrates that this approach preserves the operational characteristics in a more realistic scenario where the true response rate of the HC group is unknown. We also show that the controlling percentiles can be used to describe the joint behavior of the power and type I error. It provides a new perspective on the assessment of HCTs.

Sample size considerations for historical control studies with survival outcomes

“…Instead of the means of conditional powers and type I errors over hypothetical repetitions, a trial characteristic more relevant to researchers might be: what is the probability of actually achieving the desired power and type I error in this trial? Zhang et al (2010) and (2013) suggested a different sample size approach such that the operational characteristics (conditional power and type I error) of an HCT will be more desirable than nominal levels with pre-specified probabilities. To be specific, they proposed to control the conditional power and type I error by their percentiles instead of means.…”

“…Zhang et al (2010) and (2013) developed sample size formulas that control the percentiles of the conditional power and type I error for HC studies with continuous and binary outcomes. In this study we propose to extend this approach to HC studies with survival outcomes.…”