Online control of the false discovery rate in biomedical research

Abstract

Modern biomedical research frequently involves testing multiple related hypotheses, while maintaining control over a suitable error rate. In many applications the false discovery rate (FDR), which is the expected proportion of false positives among the rejected hypotheses, has become the standard error criterion. Procedures that control the FDR, such as the well-known Benjamini-Hochberg procedure, assume that all p-values are available to be tested at a single time point. However, this ignores the sequential nature of many biomedical experiments, where a sequence of hypotheses is tested without having access to future p-values or even the number of hypotheses. Recently, the first procedures that control the FDR in this online manner have been proposed by Javanmard

Dynamic Algorithms for Online Multiple Testing

“…For instances of application, Xu et al (2015) provide a comprehensive overview of how online hypothesis testing plays a crucial in modern A/B testing. Robertson and Wason (2018); Robertson et al (2019) apply the problem in the biomedical setting and produce a R package for online hypothesis testing.…”

Online error rate control for platform trials

“…Online error rate control methodology provides a possible solution to the problem of multiplicity for platform trials where multiple treatments are tested over time. 17,18 In the online multiple hypothesis testing framework, hypotheses are tested one-by-one over time, where at each time-step a decision is made whether to reject the current null hypothesis without knowledge of future hypothesis tests but based solely on past decisions. Methodology has recently been developed for online control of the false discovery rate (FDR) [19][20][21][22][23][24] as well as the familywise error rate (FWER), 25 so that the relevant error rate is controlled at all times throughout the trial.…”

Online error control for platform trials

“…Online error control methodology provides a possible solution to the problem of multiplicity for platform trials where multiple treatments are tested over time [17]. In the online testing framework, hypotheses are tested in a sequential manner, where at each time-step a decision is made whether to reject the current null hypothesis without knowledge of future tests but based solely on past decisions.…”