A two-step hierarchical hypothesis set testing framework, with applications to gene expression data on ordered categories

Abstract

BackgroundIn complex large-scale experiments, in addition to simultaneously considering a large number of features, multiple hypotheses are often being tested for each feature. This leads to a problem of multi-dimensional multiple testing. For example, in gene expression studies over ordered categories (such as time-course or dose-response experiments), interest is often in testing differential expression across several categories for each gene. In this paper, we consider a framework for testing multiple sets of hypothesis, which can be applied to a wide range of problems.ResultsWe adopt the concept of the overall false discovery rate (OFDR) for controlling false discoveries on the hypothesis set level. Based on an existing procedure for identifying differentially expressed gene sets, we discuss a general two-step hierarchical hypothesis set testing procedure, which controls the overall false discovery rate under independence across hypothesis sets. In addition, we discuss the concept of the mixed-directional false discovery rate (mdFDR), and extend the general procedure to enable directional decisions for two-sided alternatives. We applied the framework to the case of microarray time-course/dose-response experiments, and proposed three procedures for testing differential expression and making multiple directional decisions for each gene. Simulation studies confirm the control of the OFDR and mdFDR by the proposed procedures under independence and positive correlations across genes. Simulation results also show that two of our new procedures achieve higher power than previous methods. Finally, the proposed methodology is applied to a microarray dose-response study, to identify 17 β-estradiol sensitive genes in breast cancer cells that are induced at low concentrations.ConclusionsThe framework we discuss provides a platform for multiple testing procedures covering situations involving two (or potentially more) sources of multiplicity. The framework is easy to use and adaptable to various practical settings that frequently occur in large-scale experiments. Procedures generated from the framework are shown to maintain control of the OFDR and mdFDR, quantities that are especially relevant in the case of multiple hypothesis set testing. The procedures work well in both simulations and real datasets, and are shown to have better power than existing methods.

Two-Step Mixed Model Approach to Analyzing Differential Alternative RNA Splicing

“…These approaches usually lack statistical power due to the large number of hypothesis tests performed at the same time. A two-stage procedure was proposed to test differentially expressed gene sets [26], which was later generalized to a two-step hierarchical hypothesis set testing framework in microarray time-course experiments [32]. A similar two-step approach (stageR) was proposed to control the gene level false discovery rate (FDR) and…”

Two-step mixed model approach to analyzing differential alternative RNA splicing

“…Utilizing a two-step hierarchical hypothesis-testing framework, we will control for the overall false discovery rate (OFDR) at the gene level, which was recommended over FDR because it focuses on the inferential units of interest [26]. Compared with standard approaches that perform the hypothesis tests on all isoforms simultaneously, the number of tests performed in the proposed framework will be dramatically reduced, and consequently the statistical power is expected to be increased [32].…”

“…It was proved that the above procedure controls the OFDR at level α under the condition that the individual hypothesis tests in the second confirmatory stage are independent from all other screening tests [26,32].…”

Bayesian Genome- and Epigenome-Wide Association Studies with Gene Level Dependence

“…To compare the Bayesian methods above with frequentist methods for multiple hypothesis testing, we compute a p‐value for the null using Fisher's exact test at each marker. We consider different multiplicity adjustments for these p‐values, including (5) separate false‐discovery rate (FDR) corrections for each gene, using the Benjamini–Hochberg method (Benjamini and Hochberg, ), (6) an overall FDR correction for all markers, and (7) a two‐step hierarchical hypothesis testing framework (Li and Ghosh, ) that uses the Hochberg (Hochberg, ) and Benjamini–Hochberg methods. The latter method controls the overall FDR while allowing for dependence within sets of hypotheses.…”