Bayesian multivariate probability of success using historical data with type I error rate control

Abstract

Summary In clinical trials, it is common to have multiple clinical outcomes (e.g., coprimary endpoints or a primary and multiple secondary endpoints). It is often desirable to establish efficacy in at least one of multiple clinical outcomes, which leads to a multiplicity problem. In the frequentist paradigm, the most popular methods to correct for multiplicity are typically conservative. Moreover, despite guidance from regulators, it is difficult to determine the sample size of a future study with multiple clinical outcomes. In this article, we introduce a Bayesian methodology for multiple testing that asymptotically guarantees type I error control. Using a seemingly unrelated regression model, correlations between outcomes are specifically modeled, which enables inference on the joint posterior distribution of the treatment effects. Simulation results suggest that the proposed Bayesian approach is more powerful than the method of Holm (1979), which is commonly utilized in practice as a more powerful alternative to the ubiquitous Bonferroni correction. We further develop multivariate probability of success, a Bayesian method to robustly determine sample size in the presence of multiple outcomes.

A Bayesian approach to study design and analysis with type I error rate control for response variables of mixed types

“…For example, the value of $$$ {a}_0 $$$ that minimizes the deviance information criterion (DIC) 29 may be used. When designing a study, it has been recommended to try multiple values of $$$ {a}_0 $$$ that balance the tradeoff between power and type I error inflation 5,28 …”

“…When designing a study, it has been recommended to try multiple values of a 0 that balance the tradeoff between power and type I error inflation. 5,28…”

“…We further show how the GCGLM may be utilized to determine sample size using a multivariate version of probability of success (POS), 5 which is also called assurance 6 . The proposed method enables practitioners to plan a study that yields, for example, a high probability to reject a null hypothesis corresponding to a parameter of primary interest and at least one null hypothesis corresponding to multiple parameters of secondary interest.…”

Rejoinder to Discussions on “Optimal Test Procedures for Multiple Hypotheses Controlling the Familywise Expected Loss”

“…LaVange et al ( 2023) and Banerjee ( 2023) advocated the use of Bayesian methods as an alternative approach to classical decision theory. LaVange et al ( 2023) discussed a simulation-based framework for Bayesian designs relying on recent developments by Alt et al (2022) for FWER control. Banerjee (2023) focused mainly on Bayesian modeling for complex spatial data and followed the approach by Müller et al (2004) for FDR control.…”

“…Despite this link, one essential difference remains regarding the incorporation of weights into the error rate control. Both Alt et al (2022) and Banerjee (2023) used decision rules based on thresholding relevant posterior probabilities. To be more precise, Alt et al (2022) compared the posterior probability of at least one incorrect rejection under the global null hypothesis of no overall treatment effect with an adjusted threshold 𝜙 0 which is derived from the asymptotic distribution of this posterior probability in order to achieve FWER control.…”