False discovery rate control for multiple testing based on discrete p‐values

Abstract

For multiple testing based on discrete p‐values, we propose a false discovery rate (FDR) procedure “BH+” with proven conservativeness. BH+ is at least as powerful as the BH (i.e., Benjamini‐Hochberg) procedure when they are applied to superuniform p‐values. Further, when applied to mid‐p‐values, BH+ can be more powerful than it is applied to conventional p‐values. An easily verifiable necessary and sufficient condition for this is provided. BH+ is perhaps the first conservative FDR procedure applicable to mid‐p‐values and to p‐values with general distributions. It is applied to multiple testing based on discrete p‐values in a methylation study, an HIV study and a clinical safety study, where it makes considerably more discoveries than the BH procedure. In addition, we propose an adaptive version of the BH+ procedure, prove its conservativeness under certain conditions, and provide evidence on its excellent performance via simulation studies.

A weighted FDR procedure under discrete and heterogeneous null distributions

“…Further, in our simulation studies, we have observed considerable power improvement of the wFDR procedure over the procedures of Habiger (2015), Döhler et al. (2018), and Chen (2019). However, it is challenging to derive simple, analytic conditions for which the wFDR procedure has no less rejections than these three procedures because the former uses discrete p ‐values and the latter three use randomized p ‐values, a complicated step‐up critical sequence, or mid p ‐values, respectively.…”

“…(2018), the procedure (BHH) of Heyse (2011), the randomized Storey procedure (SR) of Habiger (2015) (with tuning parameter because other methods such as “smoother” and “bootstrap” provided by the package to determine seems to make the resulting SR a bit anticonservative in the simulation studies to be presented next), the “aHSU” procedure of Döhler et al. (2018), and the “BH+” procedure of Chen (2019). The BH+ procedure is proven by Chen (2019) to be conservative and will be applied to two‐sided mid‐ p ‐values and be denoted by “BH+MidP”; a definition of mid‐ p ‐value can be found in Hwang and Yang (2001).…”

“…(2018), and the “BH+” procedure of Chen (2019). The BH+ procedure is proven by Chen (2019) to be conservative and will be applied to two‐sided mid‐ p ‐values and be denoted by “BH+MidP”; a definition of mid‐ p ‐value can be found in Hwang and Yang (2001). For the simulated data, there do not exist groups of hypotheses for which each group has its own proportion of true null hypotheses, and condition (3) in Theorem 2 is not necessarily satisfied.…”

Multiple Comparison Procedures for Discrete Uniform and Homogeneous Tests

“…Indeed, the adjusted discrete p ‐values of Heller and Gur (2012) and Heyse (2011) reduce to the ones for continuous p ‐values in Benjamini and Hochberg (1995) and Benjamini and Yekutieli (2001), respectively, when applied to hdu p ‐values, leaving the results unchanged. The same holds true for the method of Chen (2020). Therefore, we decide to focus our research on the q ‐value approach proposed by Storey (2003) but based on estimators of the proportion of true null hypotheses, , which take the discreteness of the p ‐values into account.…”

“…More precisely, they proved the FDR control of the BH procedure applied to two‐sided mid p ‐values of Binomial tests and Fisher's exact tests. In the same line of research, Chen (2020) proposed a new BH procedure which controls the FDR when applied to mid‐ p ‐values and to p ‐values with general distributions.…”