Multiple testing with discrete data: Proportion of true null hypotheses and two adaptive FDR procedures

Abstract

We consider multiple testing with false discovery rate (FDR) control when p values have discrete and heterogeneous null distributions. We propose a new estimator of the proportion of true null hypotheses and demonstrate that it is less upwardly biased than Storey's estimator and two other estimators. The new estimator induces two adaptive procedures, that is, an adaptive Benjamini-Hochberg (BH) procedure and an adaptive Benjamini-Hochberg-Heyse (BHH) procedure. We prove that the adaptive BH (aBH) procedure is conservative nonasymptotically. Through simulation studies, we show that these procedures are usually more powerful than their nonadaptive counterparts and that the adaptive BHH procedure is usually more powerful than the aBH procedure and a procedure based on randomized p-value. The adaptive procedures are applied to a study of HIV vaccine efficacy, where they identify more differentially polymorphic positions than the BH procedure at the same FDR level.

A weighted FDR procedure under discrete and heterogeneous null distributions

“…Further, we refer to a p-value as "super-uniform" if its CDF satisfies ( ) ≤ for all ∈ [0, 1]. The following result justifies the nonasymptotic conservativeness of the wFDR procedure, and provides a condition under which the wFDR procedure nonasymptotically has no less rejections than the "aBH procedure" of Chen et al (2018).…”

“…When p-values are super-uniform under null hypotheses and are independent, we prove that the wFDR procedure is conservative nonasymptotically. Under the same setting, we provide in Appendix B a nonasymptotic upper bound on the FDR of the wFDR procedure when the weights are constructed using the proportion estimator of Chen et al (2018). This bound also applies to the FDR of the adaptive GBH procedure of Hu et al (2010), and is perhaps the best such bound.…”

“…This bound also applies to the FDR of the adaptive GBH procedure of Hu et al (2010), and is perhaps the best such bound. A sufficient condition is given such that the wFDR procedure is more powerful than the "adaptive BH procedure" (aBH) in Chen et al (2018). For MT based on p-values of BTs and FETs, we empirically show that the wFDR procedure is much more powerful than this adaptive procedure, the "randomized Storey procedure" of Habiger (2015) and four other procedures, and that the stability of the wFDR procedure is robust to the number of groups that the hypotheses are partitioned into and the value of its tuning parameter.…”

“…We provide two applications of the wFDR procedure to MT based on discrete data, one for drug safety study and the other differential methylation study, where p-values have discrete and heterogeneous null distributions. The wFDR procedure will be implemented with = 0.5 (based on our experience with MT with discrete p-values) and the grouping strategy in Step 3 of the simulation design in Section 3.1, and it will be compared with the BH procedure, "aBH" procedure of Chen et al (2018) and the "aHSU" procedure of Döhler et al (2018). The proportion of true null hypotheses in each group and that for all hypotheses will be estimated by the estimator of Chen et al (2018).…”

“…| | and eacĥ0 is given by the estimator̂0 in Chen et al (2018) as an estimate of the proportion * 0 of true null hypotheses among the th group of null hypotheses, then we call this version of the wFDR procedure "plug-in wFDR procedure," which is also an adaptive GBH of Hu et al (2010). Note that the plug-in wFDR procedure makes no rejections when * 0 = 1, and that̂0 is "reciprocally conservative" as justified by Theorem 2 of Chen et al (2018). The assumptions we need are stated as follows:…”