Design and analysis of a clinical trial using previous trials as historical control

Abstract

Background/Aims For single arm trials, a treatment is evaluated by comparing an outcome estimate to historically reported outcome estimates. Such a historically controlled trial is often analyzed as if the estimates from previous trials were known without variation and there is no trial-to-trial variation in their estimands. We develop a test of treatment efficacy and sample size calculation for historically controlled trials that considers these sources of variation. Methods We fit a Bayesian hierarchical model, providing a sample from the posterior predictive distribution of the outcome estimand of a new trial, which, along with the standard error of the estimate, can be used to calculate the probability that the estimate exceeds a threshold. We then calculate criteria for statistical significance as a function of the standard error of the new trial and calculate sample size as a function of difference to be detected. We apply these methods to clinical trials for amyotrophic lateral sclerosis using data from the placebo groups of 16 trials. Results We find that when attempting to detect the small to moderate effect sizes usually assumed in amyotrophic lateral sclerosis clinical trials, historically controlled trials would require a greater total number of patients than concurrently controlled trials, and only when an effect size is extraordinarily large is a historically controlled trial a reasonable alternative. We also show that utilizing patient level data for the prognostic covariates can reduce the sample size required for a historically controlled trial. Conclusion This article quantifies when historically controlled trials would not provide any sample size advantage, despite dispensing with a control group.

Arsenic Combined With All-Trans Retinoic Acid for Pediatric Acute Promyelocytic Leukemia: Report From the CCLG-APL2016 Protocol Study

“…One limitation of our study is that this is a single-arm trial, and therefore, the grade of evidence is not as high as that obtained from a randomized controlled trial 33 ; however, childhood APL is very rare, 34 and the excellent efficacy of arsenic plus ATRA for adult patients with APL has been well-confirmed. This is why our study was designed to be a single-arm trial in children with APL as recommended by experts in our clinical epidemiology and evidence-based medicine center.…”

Historical Controls in Randomized Clinical Trials: Opportunities and Challenges

“…Schoenfeld et al 94 . used a Bayesian hierarchical model to examine and compare the total sample sizes needed using historical vs. concurrent controls in the setting of trials of amyotrophic lateral sclerosis.…”

“…93 Cuff also demonstrated through simulations that either standard or more conservative weighting of historical data may reduce power compared with ignoring historical data. 92 Schoenfeld et al 94 used a Bayesian hierarchical model to examine and compare the total sample sizes needed using historical vs. concurrent controls in the setting of trials of amyotrophic lateral sclerosis. They found that using only historical controls did not offer an advantage in total sample size over a two-armed RCT when accounting for between-trial heterogeneity.…”

Challenges of using external data in clinical trials- an illustration in patients with COVID-19

“…The process of borrowing external data uses various Bayesian approaches that primarily differ in the assumptions regarding the relevance and exchangeability of the external data with the current trial [ 3 – 5 ]. They typically use power priors, down-weighting controls using fixed weights [ 3 , 6 ], or more complex meta-analytic priors such as normalized power prior [ 7 ], commensurate priors [ 4 ], robust meta analytic-predictive priors (MAP) [ 8 , 9 ], and supervised methods that manually adjust the informativeness of the prior based on measures of conflict between the prior information and the new trial data, assessed at the time of the analysis. Such a variety of proposed approaches was illustrated in a review that identified 58 Bayesian and 44 frequentist methods for incorporating historical data into a contemporary trial based on 52 articles [ 10 ].…”