Control of Type I Error Rates in Bayesian Sequential Designs

Shi, Haolun; Yin, Guosheng

doi:10.1214/18-ba1109

Cited by 28 publications

(47 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[28] to provide Bayesian designs with good frequentist properties. Whilst informative priors are useful in smaller early phase studies, where higher alpha levels are generally permitted, they are rarely used in confirmatory trials since they can lead to increased type I error rates if additional adjustments are not made to the stopping boundaries (e.g., [14,23]). Kopp-Schneider et al [29] demonstrated that strict control of type I error implies that no power gain is possible under any mechanism of incorporation of prior information.…”

Section: Discussionmentioning

confidence: 99%

“…One could also control for type I error in designs that allow for early stopping for efficacy by using more stringent stopping boundaries for earlier interim analyses, as we demonstrated in the HR case study. Shi and Yin [23] showed that through careful calibration of the stopping boundaries, Bayesian sequential designs can maintain the frequentist type I and II error rates at the nominal levels.…”

Section: Discussionmentioning

confidence: 99%

“…Simulation-based approaches have often been used to tune stopping boundaries in Bayesian adaptive designs to ensure acceptable type I error, e.g., 2.5% 1-sided type I error or 5% 2sided type I error (see for example [18][19][20][21]). Theoretical and numerical approaches have also been proposed to find the optimal stopping boundaries for Bayesian adaptive designs which control the type I error through use of alpha-spending functions (e.g., [8,9,[22][23][24]).…”

Section: Current Practice For Bayesian Rctsmentioning

confidence: 99%

See 2 more Smart Citations

Do we need to adjust for interim analyses in a Bayesian adaptive trial design?

Ryan

Brock

Gates

et al. 2020

Preprint

View full text Add to dashboard Cite

Background: Bayesian adaptive methods are increasingly being used to design clinical trials and offer several advantages over traditional approaches. Decisions at analysis points are usually based on the posterior distribution of the treatment effect. However, there is some confusion as to whether control of type I error is required for Bayesian designs as this is a frequentist concept.Methods: We discuss the arguments for and against adjusting for multiplicities in Bayesian trials with interim analyses. With two case studies we illustrate the effect of including interim analyses on type I/II error rates in Bayesian clinical trials where no adjustments for multiplicities are made. We propose several approaches to control type I error, and also alternative methods for decision-making in Bayesian clinical trials.Results: In both case studies we demonstrated that the type I error was inflated in the Bayesian adaptive designs through incorporation of interim analyses that allowed early stopping for efficacy and without adjustments to account for multiplicity. Incorporation of early stopping for efficacy also increased the power in some instances. An increase in the number of interim analyses that only allowed early stopping for futility decreased the type I error, but also decreased power. An increase in the number of interim analyses that allowed for either early stopping for efficacy or futility generally increased type I error and decreased power.Conclusions: Currently, regulators require demonstration of control of type I error for both frequentist and Bayesian adaptive designs, particularly for late-phase trials. To demonstrate control of type I error in Bayesian adaptive designs, adjustments to the stopping boundaries are usually required for designs that allow for early stopping for efficacy as the number of analyses increase. If the designs only allow for early stopping for futility then adjustments to the stopping boundaries are not needed to control type I error. If one instead uses a strict Bayesian approach, which is currently more accepted in the design and analysis of exploratory trials, then type I errors could be ignored and the designs could instead focus on the posterior probabilities of treatment effects of clinically-relevant values.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Current Practice For Bayesian Rctsmentioning

confidence: 99%

See 1 more Smart Citation

Do we need to adjust for interim analyses in a Bayesian adaptive trial design?

Ryan

Brock

Gates

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…to protect power). The decision threshold thus relates the Type I error and required sample size via the number of interim analyses (Shi & Yin, 2019). Limiting the number of decisions is key to efficiently designing an adaptive trial (Jennison & Turnbull, 1999).…”

Section: Type I Errormentioning

confidence: 99%

Small Sample Size Solutions

Schoot¹,

Miočević²

2020

View full text Add to dashboard Cite

Researchers often have difficulties collecting enough data to test their hypotheses, either because target groups are small or hard to access, or because data collection entails prohibitive costs. Such obstacles may result in data sets that are too small for the complexity of the statistical model needed to answer the research question. This unique book provides guidelines and tools for implementing solutions to issues that arise in small sample research. Each chapter illustrates statistical methods that allow researchers to apply the optimal statistical model for their research question when the sample is too small. This essential book will enable social and behavioral science researchers to test their hypotheses even when the statistical model required for answering their research question is too complex for the sample sizes they can collect. The statistical models in the book range from the estimation of a population mean to models with latent variables and nested observations, and solutions include both classical and Bayesian methods. All proposed solutions are described in steps researchers can implement with their own data and are accompanied with annotated syntax in R. The methods described in this book will be useful for researchers across the social and behavioral sciences, ranging from medical sciences and epidemiology to psychology, marketing, and economics.

show abstract

“…Zhu et al . () and Shi and Yin () developed methods to control the overall type I error rate in their Bayesian group sequential test, but only in the scenario of two treatment arms.…”

Section: Introductionmentioning

confidence: 99%

A Bayesian Group Sequential SmallnSequential Multiple-Assignment Randomized Trial

Chao

Thomas

Roy

et al. 2020

Journal of the Royal Statistical Society Series C: Applied Statistics

View full text Add to dashboard Cite

A small n, sequential, multiple-assignment, randomized trial (called 'snSMART') is a small sample multistage design where participants may be rerandomized to treatment on the basis of intermediate end points. This design is motivated by the 'A randomized multicenter study for isolated skin vasculitis' trial (NCT02939573): an on-going snSMART design focusing on the evaluation of three drugs for isolated skin vasculitis. By formulating an interim decision rule for removing one of the treatments, we use a Bayesian model and the resulting posterior distributions to provide sufficient evidence that one treatment is inferior to the other treatments before enrolling more participants. By doing so, we can remove the worst performing treatment at an interim analysis and prevent the subsequent participants from receiving the removed treatment. On the basis of simulation results, we have evidence that the treatment response rates can still be unbiasedly and efficiently estimated in our new design, especially for the treatments with higher response rates. In addition, by adjusting the decision rule criteria for the posterior probabilities, we can control the probability of incorrectly removing an effective treatment.

show abstract

Control of Type I Error Rates in Bayesian Sequential Designs

Cited by 28 publications

References 38 publications

Do we need to adjust for interim analyses in a Bayesian adaptive trial design?

Do we need to adjust for interim analyses in a Bayesian adaptive trial design?

Small Sample Size Solutions

A Bayesian Group Sequential SmallnSequential Multiple-Assignment Randomized Trial

Contact Info

Product

Resources

About