Increasing the efficiency of randomized trial estimates via linear adjustment for a prognostic score

Abstract

Estimating causal effects from randomized experiments is central to clinical research. Reducing the statistical uncertainty in these analyses is an important objective for statisticians. Registries, prior trials, and health records constitute a growing compendium of historical data on patients under standard-of-care conditions that may be exploitable to this end. However, most methods for historical borrowing achieve reductions in variance by sacrificing strict type-I error rate control. Here, we propose a use of historical data that exploits linear covariate adjustment to improve the efficiency of trial analyses without incurring bias. Specifically, we train a prognostic model on the historical data, then estimate the treatment effect using a linear regression while adjusting for the trial subjects' predicted outcomes (their prognostic scores). We prove that, under certain conditions, this prognostic covariate adjustment procedure attains the minimum variance possible among a large class of estimators. When those conditions are not met, prognostic covariate adjustment is still more efficient than raw covariate adjustment and the gain in efficiency is proportional to a measure of the predictive accuracy of the prognostic model. We demonstrate the approach using simulations and a reanalysis of an Alzheimer's Disease clinical trial and observe meaningful reductions in mean-squared error and the estimated variance. Lastly, we provide a simplified formula for asymptotic variance that enables power and sample size calculations that account for the gains from the prognostic model for clinical trial design.

Modeling Disease Progression in Mild Cognitive Impairment and Alzheimer's Disease with Digital Twins

“…A typical clinical trial compares the safetey and efficacy of an investigational therapy to a placebo and standard-of-care. Therefore, there are many uses for models that can generate accurate forecasts for disease progression under standard-of-care ranging from clinical trial design to statistical analysis plans that directly incorporate prognostic forecasts to improve statistical power [19,26]. Here, we built on previous work by training a generative model to forecast disease progression for subjects with AD, aiming to obtain well-calibrated forecasts across the spectrum of severity from early to severe disease.…”

“…Similarly, conditional generative models can be used to generate distributions of potential control outcomes for individual patients ( [7,26,27,31]), which we call 'Digital Twins', that can be incorporated into clinical trials to increase statistical power or reduce required sample sizes [19,26].…”

On the relevance of prognostic information for clinical trials: A theoretical quantification

“…The predicted variance reduction for the trial, following Borm et al (2007) and also more recently Branders et al (2021) and Schuler et al (2021), would then be 1 − R 2 OOS .…”

“…Equivalently, for fixed sample size n the precision of the treatment effect estimate increases as the residual variance decreases. It should be noted that the classical "design factor" 1 − R 2 OOS (Borm et al 2007;Branders et al 2021;Schuler et al 2021) is biased in our setup, because 1…”

“…In clinical practice, prognostic scores aim to provide a tool for risk stratification, for example for clinical behaviour of a disease (e.g., in prostate cancer, Kreuz et al 2020) or in the intensive care unit (e.g., the APACHE or FOUR scores, Knaus et al 1991;Wijdicks et al 2005). Statistical methods relying on prognostic scores (i.e., disease risk scores for binary outcomes), are widely employed for observational studies (Nguyen et al 2020;Aikens et al 2020;Wyss et al 2016;Ray 2011, 2009) and have since also found application in clinical trials, e.g., for stratification (Cellini et al 2019;Hurwitz et al 2018;Herrera et al 2020;Saffi et al 2014) or covariate adjustment (Schuler et al 2021;Branders et al 2021).…”