Controlling the false discovery exceedance for heterogeneous tests

Döhler, Sebastian; Roquain, Étienne

doi:10.1214/20-ejs1771

Cited by 6 publications

(5 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Poisson's Binomial Distributions or PBDs have been used in the FDR literature, more recently by Döhler and Roquain (2020) in the context of controlling the FDX for heterogeneous tests. In the two-group model, each hypothesis is true either under the null or the alternative with probability 1 − π and π.…”

Section: Poisson Binomial Distribution and Connection To Lfdrmentioning

confidence: 99%

An Empirical Bayes Approach to Controlling the False Discovery Exceedance

Basu¹,

Fu²,

Saretto³

et al. 2021

Preprint

View full text Add to dashboard Cite

In sparse large-scale testing problems where the false discovery proportion (FDP) is highly variable, the false discovery exceedance (FDX) provides a valuable alternative to the widely used false discovery rate (FDR). We develop an empirical Bayes approach to controlling the FDX. We show that for independent hypotheses from a two-group model and dependent hypotheses from a Gaussian model fulfilling the exchangeability condition, an oracle decision rule based on ranking and thresholding the local false discovery rate (lfdr ) is optimal in the sense that the power is maximized subject to FDX constraint. We propose a data-driven FDX procedure that emulates the oracle via carefully designed computational shortcuts. We investigate the empirical performance of the proposed method using simulations and illustrate the merits of FDX control through an application for identifying abnormal stock trading strategies.

show abstract

Section: Poisson Binomial Distribution and Connection To Lfdrmentioning

confidence: 99%

An Empirical Bayes Approach to Controlling the False Discovery Exceedance

Basu¹,

Fu²,

Saretto³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Poisson's Binomial Distributions or PBDs have been used in the FDR literature, more recently by Döhler and Roquain (2020) in the context of controlling the FDX for heterogeneous tests.…”

Section: Poisson Binomial Distribution and Connection To Lfdrmentioning

confidence: 99%

“…We prove that the ranking is optimal (see also, Fu, 2018) and the procedure controls the FDX at the pre-specified level (see also, Basu, 2016). Our work is most similar to Döhler and Roquain (2020) who use the Poisson Binomial distribution to threshold p-values under the frequentist setting. From a theoretical standpoint, the derivation of our procedure is also similar to the theoretical work of Heller and Rosset (2021) who study FDR control in an empirical Bayes framework.…”

Section: Introductionmentioning

confidence: 99%

“…The merits of controlling the FDX have been discussed in Wasserman (2004, 2006); Guo and Romano (2007); Chi and Tan (2008); Gordon and Salzman (2008); Delattre and Roquain (2015). Different FDX methods have been proposed in Lehmann and Romano (2005), Chi and Tan (2008), Roquain and Villers (2011), and Döhler and Roquain (2020). These procedures differ in choosing their respective cut-offs for rejected tests but are all similar in using p-values to rank hypotheses.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Empirical Bayes Control of the False Discovery Exceedance

Basu¹,

Fu²,

Saretto³

et al. 2021

View full text Add to dashboard Cite

“…However, it has not been fully characterized how the dependence among the test statistics would impact FDX as opposed to FDR in the literature. In particular, some works on FDX assume independence among tests (Genovese and Wasserman, 2004;Guo and Romano, 2007;Ge and Li, 2012); more importantly, most works focus on developing procedures that control FDX as a single-value characteristic of FDP other than investigating how dependence affects the uncertainty and/or distribution of FDP (Korn et al, 2004;van der Laan et al, 2004;Lehmann and Romano, 2005;Genovese and Wasserman, 2006;Delattre and Roquain, 2015;Hemerik et al, 2019;Döhler and Roquain, 2020;Basu et al, 2021). Thus, these works still lack the valuable information about how FDP varies from study to study.…”

Section: Introductionmentioning

confidence: 99%

Asymptotic Uncertainty of False Discovery Proportion

Meng¹,

Chen²,

Jiang³

2022

Preprint

View full text Add to dashboard Cite

Multiple testing has been a popular topic in statistical research. Although vast works have been done, controlling the false discoveries remains a challenging task when the corresponding test statistics are dependent. Various methods have been proposed to estimate the false discovery proportion (FDP) under arbitrary dependence among the test statistics. One of the main ideas is to reduce arbitrary dependence to weak dependence and then to establish theoretically the strong consistency of the FDP and false discovery rate (FDR) under weak dependence. As a consequence, FDPs share the same asymptotic limit in the framework of weak dependence. We observe that the asymptotic variance of the FDP, however, may rely heavily on the dependence structure of the corresponding test statistics even when they are only weakly dependent; and it is of great practical value to quantify this variability, as it can serve as an indicator of the quality of the FDP estimate from the given data. As far as we are aware, the research on this respect is still limited in the literature. In this paper, we first derive the asymptotic expansion of FDP under mild regularity conditions and then examine how the asymptotic variance of FDP varies under different dependence structures both theoretically and numerically. With the observations in this study, we recommend that in a multiple testing performed by an FDP procedure, we may report both the mean and the variance estimates of FDP to enrich the study outcome.

show abstract

Controlling the false discovery exceedance for heterogeneous tests

Cited by 6 publications

References 40 publications

An Empirical Bayes Approach to Controlling the False Discovery Exceedance

An Empirical Bayes Approach to Controlling the False Discovery Exceedance

Empirical Bayes Control of the False Discovery Exceedance

Asymptotic Uncertainty of False Discovery Proportion

Contact Info

Product

Resources

About