Multiple hypothesis testing in genomics

Abstract

This paper presents an overview of the current state of the art in multiple testing in genomics data from a user's perspective. We describe methods for familywise error control, false discovery rate control and false discovery proportion estimation and confidence, both conceptually and practically, and explain when to use which type of error rate. We elaborate on the assumptions underlying the methods and discuss pitfalls in the interpretation of results. In our discussion, we take into account the exploratory nature of genomics experiments, looking at selection of genes before or after testing, and at the role of validation experiments.

Correlations of inflammatory gene pathways, corticolimbic functional activities, and aggression in pediatric bipolar disorder: A preliminary study

“…Since this is an exploratory study, we did not implement multi-testing corrections to identify significant association findings. Prior studies have suggested that an inflated familywise error rate (FWER) due to a lack of multi-testing corrections may be acceptable in an exploratory study, since the hypothesis would need to be addressed a priori and cannot be discerned by any a posteriori analysis (Goeman and Solari, 2014; Stacey et al, 2012). …”

When null hypothesis significance testing is unsuitable for research: a reassessment

“…So Q can be considered a random variable. However, as Q cannot be controlled directly FDR is defined as the expected value of the proportion of false positive errors: FDR = E[FP/R|R > 0] · pr(R > 0), a variable which can be controlled (see Benjamini and Hochberg, 1995;Curran-Everett, 2000;Nichols and Hayasaka, 2003;Bennett et al, 2009;Benjamini, 2010;Goeman and Solari, 2014). Some FDR estimation procedures can also factor in dependency between tests (Benjamini and Yekutieli, 2001).…”

Comment on: Epidermolysis Bullosa Pruriginosa: A Systematic Review Exploring Genotype–Phenotype Correlation

“…Indeed, although some authors have suggested that the Bonferroni procedure only addresses the ''universal null hypothesis'' and not the individual hypotheses, that claim is unequivocally false [4,5] and does not appear in the statistical literature. Armstrong's source for the ''universal null hypothesis'' misconception is an opinion piece [6] that has been discredited both by statisticians [5] and by other researchers [7,8], largely for that very misconception. Armstrong's paper, like that of Perneger [6], does not support its erroneous statements with any theorems, proofs, simulations, citations of statistical literature, or any other non-anecdotal evidence (in fact, the one formula it contains is incorrect 1 ).…”

“…Any hypotheses worth testing-planned or not-have a non-zero probability of Type I error; the greater the number of tests in a given study, the greater that probability becomes [4,5,9,10]. Unpredicted significance may demand more skepticism than predicted significance, but ''multiple tests are multiple tests, whether planned in advance or not'' [11].…”