Ignorance is not bliss: Statistical power is not probability of trial success

Zierhut, ML; Bycott, Paul; Gibbs, MA; Bp, Smith; Vicini, Paolo

doi:10.1002/cpt.257

Cited by 7 publications

(10 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Data on the comparator/competitor landscape are often available from previously conducted trials. As such, development of an MBMA framework that integrates these data from across trials allows in silico simulations of potential phase III trial designs with different comparators to forecast probability of success as a function of trial design (e.g., selected comparator, patient population features) . Probability of success of a trial can in turn inform the probability of technical success for a phase III program and, along with economics data, provide a risk‐adjusted estimate of the economic value of each asset in the portfolio at this critical inflection point in development.…”

Section: Examples Of Strategic Integration Of Mbma To Inform Key Decimentioning

confidence: 99%

“…These models can serve as priors to quantify how an asset under development benchmarks to the current standard of care, other internal pipeline assets, and external competitor pipeline assets under development. They enable model‐informed portfolio prioritization and decision analysis in the context of in‐licensing considerations, thereby contributing to objective estimation of probability of success and principled drug development decision making . This is especially critical at the end of phase II milestone for informing go/no go decisions to proceed to phase III.…”

Section: Examples Of Strategic Integration Of Mbma To Inform Key Decimentioning

confidence: 99%

“…As such, development of an MBMA framework that integrates these data from across trials allows in silico simulations of potential phase III trial designs with different comparators to forecast probability of success as a function of trial design (e.g., selected comparator, patient population features). 7 Probability of success of a trial can in turn inform the probability of technical success for a phase III program and, along with economics data, provide a risk-adjusted estimate of the economic value of each asset in the portfolio at this critical inflection point in development. This can help guide portfolio decisions by ranking the risk-adjusted value of different compounds competing for a fixed amount of resources and providing a view of the overall risk/uncertainty in the phase III portfolio.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Model‐Based Meta‐Analysis: Optimizing Research, Development, and Utilization of Therapeutics Using the Totality of Evidence

Upreti

Venkatakrishnan

2019

Clin Pharma and Therapeutics

View full text Add to dashboard Cite

Model‐based meta‐analysis (MBMA) is a valuable component of the quantitative pharmacology toolkit for model‐informed drug discovery and development. It enables principled decision making with a totality of evidence mindset through integration of internal and external data across multiple dimensions (e.g., targets/mechanisms, molecules/drugs, doses/regimens, diseases/indications, populations, endpoints, and clinical trial designs). MBMA distinguishes itself from traditional meta‐analysis by infusing pharmacologic plausibility into the statistical rigor that typifies meta‐analytic data integration. This is possible through mechanism‐informed formulation of pharmacologically inspired cause–effect and dose‐response relationships, time course of treatment effects, and interrelationships between proximal and distal outcomes of modulation of disease biology and pathophysiology. In this review, we offer a question‐based approach to enhance appreciation of the value of MBMA across the continuum from drug discovery and translational research through clinical development, comparative effectiveness research, and postapproval optimization of therapeutics using illustrative examples across therapeutic areas.

show abstract

Section: Examples Of Strategic Integration Of Mbma To Inform Key Decimentioning

confidence: 99%

Section: Examples Of Strategic Integration Of Mbma To Inform Key Decimentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Model‐Based Meta‐Analysis: Optimizing Research, Development, and Utilization of Therapeutics Using the Totality of Evidence

Upreti

Venkatakrishnan

2019

Clin Pharma and Therapeutics

View full text Add to dashboard Cite

show abstract

“…Liu (2010) extended the BEP by accounting for the uncertainty around both µ and σ (or the effect size ∆), and also added two more versions of the BEP to by removing the "type I error" component from the regular BEP metric. Some recent reviews, discussion, and applications of the BEP in early and late phased clinical trials, and in meta-analysis are given in Kirby et al (2012); Carroll (2013); Ibrahim et al (2015); Du and Wang (2016); Zierhut et al (2016). The BEP has been routinely calculated alongside the traditional power in some pharmaceutical companies; and it is also implemented in several sample size and power calculation software (Labes et al, 2016;EAST-CYTEL, 2016).…”

Section: Introductionmentioning

confidence: 99%

Assessment of Bayesian expected power via Bayesian bootstrap

Liu

2018

Statistics in Medicine

View full text Add to dashboard Cite

The Bayesian expected power (BEP) has become increasingly popular in assessing the probability of success for a future trial. While the traditional power assumes a single value for the unknown effect size Δ and is thus highly dependent on the assumed value, the BEP embraces the uncertainty around Δ given the prior information and is therefore a less subjective measure for the probability of success than the traditional power especially when the prior information is not rich. Current methods for assessing BEP are often based in a parametric framework by imposing a model on the pilot data to derive and sample from the posterior distributions of Δ. The model-based approach can be analytically challenging and computationally costly especially for multivariate data sets, and it also runs the risk of generating misleading BEP if the model is misspecified. We propose an approach based on the Bayesian bootstrap (BBS) technique to simulate future trials in the presence of individual-level pilot data, based on which the empirical BEP can be calculated. The BBS approach is model-free with no assumptions about the distribution of the prior data and also circumvents the analytical and computational complexity associated with obtaining the posterior distribution of the Δ. Information from multiple pilot studies is also straightforward to combine. We also propose the double bootstrap technique, a frequentist counterpart to the BBS, that shares similar properties and achieves the same goal as the BBS for BEP assessment. Simulation and case studies are presented to demonstrate the implementation of the BBS technique and the double bootstrap technique and to compare the BEP results with model-based approach.

show abstract

“…Thus, a proposed phase III trial may be powered at 90% but have a PoS of 60% or less. 5 These various factors contribute to the surprisingly high failure rate for insufficient efficacy in phase III. 6 Anchoring is a powerful cognitive bias that can lead people to overestimate the probability that an observed result (e.g., phase II trial) will be replicated in subsequent phase III studies.…”

mentioning

confidence: 99%