A decision-theoretic approach to Bayesian clinical trial design and evaluation of robustness to prior-data conflict

Abstract

Summary Bayesian clinical trials allow taking advantage of relevant external information through the elicitation of prior distributions, which influence Bayesian posterior parameter estimates and test decisions. However, incorporation of historical information can have harmful consequences on the trial’s frequentist (conditional) operating characteristics in case of inconsistency between prior information and the newly collected data. A compromise between meaningful incorporation of historical information and strict control of frequentist error rates is therefore often sought. Our aim is thus to review and investigate the rationale and consequences of different approaches to relaxing strict frequentist control of error rates from a Bayesian decision-theoretic viewpoint. In particular, we define an integrated risk which incorporates losses arising from testing, estimation, and sampling. A weighted combination of the integrated risk addends arising from testing and estimation allows moving smoothly between these two targets. Furthermore, we explore different possible elicitations of the test error costs, leading to test decisions based either on posterior probabilities, or solely on Bayes factors. Sensitivity analyses are performed following the convention which makes a distinction between the prior of the data-generating process, and the analysis prior adopted to fit the data. Simulation in the case of normal and binomial outcomes and an application to a one-arm proof-of-concept trial, exemplify how such analysis can be conducted to explore sensitivity of the integrated risk, the operating characteristics, and the optimal sample size, to prior-data conflict. Robust analysis prior specifications, which gradually discount potentially conflicting prior information, are also included for comparison. Guidance with respect to cost elicitation, particularly in the context of a Phase II proof-of-concept trial, is provided.

Simulating and reporting frequentist operating characteristics of clinical trials that borrow external information: Towards a fair comparison in case of one‐arm and hybrid control two‐arm trials

“…Unless one test is clearly superior in terms of positive and negative likelihood ratio, the application of this approach relies on the specific context, i.e., whether controlling size or increasing power is more important. Several additional proposals have recently tried to move away from strict control of T1E rate, for example, by giving rationales for targeting a weighted sum of T1E and T2E rates instead 19–22 . This can be accomplished by assigning weights representing the relative seriousness of T1E versus T2E rates (or relative trust on the null and alternative hypothesis), evaluated at and .…”

“…Bayesian prior probabilities can be incorporated in such an assessment; these can be used to assign a prior probability to each (point) hypothesis or to average across the support of itself. In the latter case, the costs can be reformulated so that optimal Bayesian decisions would minimize a weighted sum of average (or expected) T1E and T2E rates 19,21 …”

“…Several additional proposals have recently tried to move away from strict control of T1E rate, for example, by giving rationales for targeting a weighted sum of T1E and T2E rates instead. [19][20][21][22] This can be accomplished by assigning weights representing the relative seriousness of T1E versus T2E rates (or relative trust on the null and alternative hypothesis), evaluated at θ 0 Θ 0 and θ 1 = 2 Θ 0. 20,22 In the context of one-sided one-arm testing, the latter can be directly translated into a Bayesian integrated risk minimization problem with a two-point mass prior, giving equal probability to θ 0 and θ 1 , and assigning costs to T1E and T2E equal to the above weights.…”

“…In the latter case, the costs can be reformulated so that optimal Bayesian decisions would minimize a weighted sum of average (or expected) T1E and T2E rates. 19,21 It is important to note that for one-sided one-arm testing of location parameters in exponential families, and for a given sample size, each approach would essentially select a specific rejection probability function, so that all approaches are equivalent if T1E rate (or power) is identical. The Bayesian approach justifies T1E rate inflation by trust in the prior distribution (if it favors the alternative hypothesis), and leads in turn to a specific weighted sum of errors.…”

Robust incorporation of historical information with known type I error rate inflation

“…We shortly introduce the concepts and notation which will be used in the following. Note that a similar setup has been considered in Calderazzo et al (2022) to which we refer for additional details. We consider the situation of testing $$H_0:\theta \le \theta _0$$ versus the alternative hypothesis $$H_1:\theta > \theta _0$$.…”