The perils with the misuse of predictive power

Abstract

In early drug development, especially when studying new mechanisms of action or in new disease areas, little is known about the targeted or anticipated treatment effect or variability estimates. Adaptive designs that allow for early stopping but also use interim data to adapt the sample size have been proposed as a practical way of dealing with these uncertainties. Predictive power and conditional power are two commonly mentioned techniques that allow predictions of what will happen at the end of the trial based on the interim data. Decisions about stopping or continuing the trial can then be based on these predictions. However, unless the user of these statistics has a deep understanding of their characteristics important pitfalls may be encountered, especially with the use of predictive power. The aim of this paper is to highlight these potential pitfalls. It is critical that statisticians understand the fundamental differences between predictive power and conditional power as they can have dramatic effects on decision making at the interim stage, especially if used to re-evaluate the sample size. The use of predictive power can lead to much larger sample sizes than either conditional power or standard sample size calculations. One crucial difference is that predictive power takes account of all uncertainty, parts of which are ignored by standard sample size calculations and conditional power. By comparing the characteristics of each of these statistics we highlight important characteristics of predictive power that experimenters need to be aware of when using this approach.

Bayesian predictive power: choice of prior and some recommendations for its use as probability of success in drug development

“…If the prior approaches a point mass at θ 0 , PoS becomes either power or CP at θ 0 . The latter result is noted in passing in , p. 62] as well as again in and .…”

“…A comparison of the features of PoS and CP is undertaken in . The authors emphasize that PoS takes into account the uncertainty about the underlying true effect, parts of which are ignored by standard sample size calculations and CP.…”

“…In what follows, we will use PoS. For a discussion of the different power concepts, with an emphasis on predictive power, we refer to .…”

Group Sequential and Confirmatory Adaptive Designs in Clinical Trials

“…Here, non-monotonicity is likely to happen for small (and reasonable) v 2 such that this might cause real problems. Dallow and Fina (2011) illustrate this and warn against the misuse of predictive power.…”

“…The posterior variance can be utilized by adopting the Bayesian point of view more consequently and computing the Bayesian predictive power (Spiegelhalter et al 1986;Dmitrienko and Wang 2006;Wang 2007;Lan et al 2009). For a review of this approach, see also Dallow and Fina (2011).…”