A weighted FDR procedure under discrete and heterogeneous null distributions

Abstract

Multiple testing with false discovery rate (FDR) control has been widely conducted in the "discrete paradigm" where p-values have discrete and heterogeneous null distributions with finitely many discontinuities. However, existing FDR procedures may lose some power when applied to such p-values. We propose a weighted FDR procedure for multiple testing in the discrete paradigm that efficiently adapts to both the heterogeneity and discreteness of p-value distributions. We prove the conservativeness of the weighted FDR procedure and demonstrate that it is more powerful than several other procedures for multiple testing based on p-values of binomial test or Fisher's exact test. The weighted FDR procedure is applied to a drug safety study and a differential methylation study based on discrete data, where it makes more discoveries than the Benjamini-Hochberg procedure at the same FDR level.

A grouped, selectively weighted false discovery rate procedure

“…Theorem 2 does not require the number of groups l to be constant in m, allows the use of any non-increasing and reciprocally conservative estimator of the proportion of true nulls for each interesting group, and accounts for effects of selecting groups of hypotheses of interest. It generalizes Theorem 3 in Chen et al (2017). In particular, when all groups are of interest, Ŝ = {1, .…”

“…To better adapt to a group structure among hypotheses, we have proposed a grouped, selectively weighted FDR procedure, sGBH, that is a refinement and extension of the GBH and wFDR procedures of Hu et al (2010), Chen et al (2017) and Nandi and Sarkar (2018) and that accommodates scenarios where only a few groups are likely to be interesting. For the plug-in adaptive sGBH, we have provided simple conditions to ensure and some empirical evidence on its conservativeness, together with an FDR upper bound that quantifies the effect of estimating the interesting groups.…”

“…When p-values are uniformly distributed, Hu et al (2010) justified the asymptotic conservativeness of the adaptive GBH by assuming the convergence of various empirical processes related to p-values and the asymptotic conservativeness of each estimator of the groupwise proportion of true nulls. In contrast, without requiring any such empirical process to be convergent, we provide simple conditions on the conservativeness of the adaptive sGBH using the same strategy of Chen and Doerge (2017). The key to achieve this is a reciprocally conservative or consistent estimator of the proportion of true nulls.…”

“…. , l} can be set, and the upper bound η m on the right hand side of (7) reduces to that provided by Theorem 3 of Chen et al (2017).…”

“…For example, groups of hypotheses may have different proportions of true nulls, or statistics associated with a group of hypotheses may possess the same type of dependency structure or have similar powers. To utilize such information, methods based on hypotheses grouping and weighting (Liu et al;Basu et al;2018) or p-value grouping and weighting (Cai and Sun;Hu et al;2010;2017;Nandi and Sarkar;2018) have been developed.…”