Power-enhanced multiple decision functions controlling family-wise error and false discovery rates

Abstract

Improved procedures, in terms of smaller missed discovery rates (MDR), for performing multiple hypotheses testing with weak and strong control of the family-wise error rate (FWER) or the false discovery rate (FDR) are developed and studied. The improvement over existing procedures such as the Šidák procedure for FWER control and the Benjamini–Hochberg (BH) procedure for FDR control is achieved by exploiting possible differences in the powers of the individual tests. Results signal the need to take into account the powers of the individual tests and to have multiple hypotheses decision functions which are not limited to simply using the individual p-values, as is the case, for example, with the Šidák, Bonferroni, or BH procedures. They also enhance understanding of the role of the powers of individual tests, or more precisely the receiver operating characteristic (ROC) functions of decision processes, in the search for better multiple hypotheses testing procedures. A decision-theoretic framework is utilized, and through auxiliary randomizers the procedures could be used with discrete or mixed-type data or with rank-based nonparametric tests. This is in contrast to existing p-value based procedures whose theoretical validity is contingent on each of these p-value statistics being stochastically equal to or greater than a standard uniform variable under the null hypothesis. Proposed procedures are relevant in the analysis of high-dimensional “large M, small n” data sets arising in the natural, physical, medical, economic and social sciences, whose generation and creation is accelerated by advances in high-throughput technology, notably, but not limited to, microarray technology.

Efficient Stratified Testing Procedure for a False Discovery Rate

“…While Genovese et al (2006) propose a p-value weighting scheme, Ferkinstad et al (2008) suggest the use of external covariates by employing an empirical Bayes' approach. More recently, Pñna et al (2010) proposed employing the power function of a most powerful test function from Neyman and Pearson (1933) under the setting of simple null and simple alternative hypotheses. Using the so-called receiver operating characteristic (ROC) functions, Pñna et al's (2010) procedure strongly controls FDR and decreases FNR.…”

Classes of multiple decision functions strongly controlling FWER and FDR

“…This framework is appropriate, for instance, when dealing with discrete data or when using nonparametric rank-based decision functions. For more discussions on this matter, see [4, 25]. …”

“…Henceforth, to simplify our notation, we adopt a functional notation where δ(α) represents the statistic defined on 𝒳 according to x ↦ δ( x ; α). When we view this as a stochastic process in α, we obtain the notion of a (nonrandomized) decision process as introduced in [4], which is a stochastic process Δ = {δ(α) : α ∈ [0, 1]} where, ∀α ∈ [0, 1], δ(α) is a decision function, and such that the following conditions are satisfied.

δ(0) = 0 and δ(1) = 1 a.e.- P .

The sample paths α ↦ δ(α) are, a.e.- P , {0, 1}-valued step-functions which are nondecreasing and right-continuous.

…”

“…The power of δ m at μ m = γ m is given by $${\pi }_{m}false({\eta }_{m}false)=\Phi true[{\gamma }_m-{\Phi }^{-1}false(1-{\eta }_{m}false)true].$$ The weak family wise error rate of the associated multiple decision function is $$\mathrm{normalF}\mathrm{normalW}\mathrm{normalE}\mathrm{normalR}false({\eta }_m,m\in ℳfalse)={\mathbf{P}}_{0}true\{\sum _{m\in ℳ}{\delta }_{m}false(X_m;{\eta }_{m}false)>0true\}=1-\prod _{m\in ℳ}false[1-{\eta }_{m}false],$$ where P 0 is the probability measure associated with μ m = 0 for all m ∈ ℳ. Suppose we seek to control this weak FWER at an overall level α ∈ [0, 1], so that condition (A3) is necessarily satisfied, but at the same time maximize the overall power of the multiple decision at μ m = γ m , m = 1, 2, …, M , given by $$\text{POWER}false({\eta }_m,m\in ℳfalse)=\sum _{m\in ℳ}{\pi }_{m}false({\eta }_{m}false).$$ Then, (cf., [4]) the multiple size function { A m (α), m ∈ ℳ} should satisfy the two conditions: (i) for some λ ∈ ℜ, $$\forall m\in ℳ:false(1-A_{m}false(\alpha false)false)\frac{\varphi false[{\gamma }_m-{\Phi }^{-1}false(1-A_{m}false(\alpha false)false)false]}{\varphi false[{\Phi }^{-1}false(1-A_{m}false(\alpha false)false)false]}=\lambda ;$$ where ϕ(·) is the standard normal density function, and (ii) ∑ m ∈ℳ log[1 − A m (α)] = log(1 − α). If the effect sizes γ m ’s are not all identical, then these size functions are not identical, that is, they will depend on m .…”

“…These results will have important implications in the search for optimal multiple testing procedures that control either of these Type I error rates. The main results in this paper were motivated by those in [4], which did not deal with classes of multiple testing procedures, but instead simply focussed in developing improved FWER and FDR-controlling procedures from the Neyman-Pearson most powerful tests for each of the M pairs of hypotheses.…”