Leveraging historical data to optimize the number of covariates and their explained variance in the analysis of randomized clinical trials.

Abstract

The amount of data collected from patients involved in clinical trials is continuously growing. All baseline patient characteristics are potential covariates that could be used to improve clinical trial analysis and power. However, the limited number of patients in phases I and II studies restricts the possible number of covariates included in the analyses. In this paper, we investigate the cost/benefit ratio of including covariates in the analysis of clinical trials with a continuous outcome. Within this context, we address the long-running question “What is the optimum number of covariates to include in a clinical trial?” To further improve the benefit/cost ratio of covariates, historical data can be leveraged to pre-specify the covariate weights, which can be viewed as the definition of a new composite covariate. Here we analyze the use of a composite covariate to improve the estimated treatment effect in small clinical trials. A composite covariate limits the loss of degrees of freedom and the risk of overfitting.

On the relevance of prognostic information for clinical trials: A theoretical quantification

“…(2007) and also more recently Branders et al. (2021) and Schuler et al. (2021), would then be $$1 - \widehat{R^2}_{\rm OOS}$$.…”

“…We are interested in the setup, where one was able to obtain an estimate, $$\mathfrak {s}(\text{\boldmath $x$})= \widehat{\pi s}(\text{\boldmath $x$})$$, of $$\pi s(\text{\boldmath $x$})$$ either from the literature or from historical control data. The latter situation received some interest recently (Branders et al., 2021; Glynn et al., 2012; Schuler et al., 2021; Wyss et al., 2014). Assuming one has access to data from past trials on the same outcome Y and covariates $$\text{\boldmath $X$}$$ for control patients, $$z= 0$$, many statistical and machine learning procedures, for example, random forests and neural networks, can be used to estimate the prognostic score function from the conditional mean $$\mathfrak {s}(\text{\boldmath $x$})= \widehat{\pi s}(\text{\boldmath $x$}) = \widehat{\mathbb {E}}(Y\mid \text{\boldmath $X$}= \text{\boldmath $x$}, z= 0) - \hat{\alpha }$$ (with some estimate $$\hat{\alpha }$$ of the model's intercept parameter).…”

“…Equivalently, for fixed sample size n the precision of the treatment effect estimate increases as the residual variance decreases. It should be noted that the classical “design factor” $$1 - \widehat{R^2}_{\rm OOS}$$ (Borm et al., 2007; Branders et al., 2021; Schuler et al., 2021) is biased in our setup, because $$1 - R^2_{\rm OOS} = 1 - (2 \rho - 1) R^2 \ne 1 - \rho ^2 R^2$$. This discrepancy will be demonstrated empirically in Section 3.…”

“…We are interested in the setup, where one was able to obtain an estimate, , of either from the literature or from historical control data. The latter situation received some interest recently (Branders et al., 2021 ; Glynn et al., 2012 ; Schuler et al., 2021 ; Wyss et al., 2014 ). Assuming one has access to data from past trials on the same outcome Y and covariates for control patients, , many statistical and machine learning procedures, for example, random forests and neural networks, can be used to estimate the prognostic score function from the conditional mean (with some estimate of the model's intercept parameter).…”

“…In observational studies, Hansen (2008) formalized the concept of adjusting with respect to a prognostic score, which has since been studied for various applications (Arbogast & Ray, 2009, 2011; Hajage et al., 2017; Pfeiffer & Riedl, 2015; Wyss et al., 2016). Although the idea of adjusting with respect to historical prognostic scores in clinical trials was briefly sketched in an earlier paper by Cox (1982) and has been suggested for heterogeneous treatment effect estimation (Kent et al., 2020), an in‐depth development of this principle has been published only recently (Anonymous, 2022; Branders et al., 2021; Schuler et al., 2021, which was under consultation for qualification opinion by the European Medicines Agency [EMA]).…”