The performance of a new local false discovery rate method on tests of association between coronary artery disease (CAD) and genome-wide genetic variants

Mei, Shuyan; Karimnezhad, Ali; Forest, Marie; Bickel, David R.; Greenwood, Celia M.T.

doi:10.1371/journal.pone.0185174

“…If it is not useful to test the joint studywise hypothesis, then researchers should consider lower-order families of hypotheses and/or individual hypotheses for testing (Benjamini & Bogomolov, 2011;Efron, 2008;Fisher, 1971, p. 206;Hochberg & Tamrane, 1987, pp. 6-7;Hung & Wang, 2010;Mei et al, 2017;Rubin, 2017b). For example, in their discussion of multiple testing in microarray gene expression analysis, Yekutieli et al (2006) explained that "the set of hypotheses that is of interest to the researcher in a single study does not necessarily form a single family of hypotheses" (p. 416).…”

Section: Studywise Error Ratesmentioning

confidence: 99%

When to adjust alpha during multiple testing: a consideration of disjunction, conjunction, and individual testing

Rubin

¹

2021

View full text Add to dashboard Cite

Scientists often adjust their significance threshold (alpha level) during null hypothesis significance testing in order to take into account multiple testing and multiple comparisons. This alpha adjustment has become particularly relevant in the context of the replication crisis in science. The present article considers the conditions in which this alpha adjustment is appropriate and the conditions in which it is inappropriate. A distinction is drawn between three types of multiple testing: disjunction testing, conjunction testing, and individual testing. It is argued that alpha adjustment is only appropriate in the case of disjunction testing, in which at least one test result must be significant in order to reject the associated joint null hypothesis. Alpha adjustment is inappropriate in the case of conjunction testing, in which all relevant results must be significant in order to reject the joint null hypothesis. Alpha adjustment is also inappropriate in the case of individual testing, in which each individual result must be significant in order to reject each associated individual null hypothesis. The conditions under which each of these three types of multiple testing is warranted are examined. It is concluded that researchers should not automatically (mindlessly) assume that alpha adjustment is necessary during multiple testing. Illustrations are provided in relation to joint studywise hypotheses and joint multiway ANOVAwise hypotheses.

show abstract

“…The established method for controlling against multiple testing has been to adjust family-wise error rates (FWERs) such as using the Holm-Bonferroni method 23 . The control of FWERs has been considered too conservative and severely compromises statistical power resulting in many true loci of small effect being missed 24 . Recently local FDRs (LFDRs) have been proposed and are defined as the probability of a test result being false given the exact value of the test statistic 25 .…”

Section: Related Work and Previous Approaches To Local Threshold Detementioning

confidence: 99%

“…Recently local FDRs (LFDRs) have been proposed and are defined as the probability of a test result being false given the exact value of the test statistic 25 . The LFDR correction, through re-ranking of test statistic value, has been demonstrated to eliminate biases of the former non-local FDR (NFDR) estimators 24 , 26 . Our application is similarly motivated, though differs in that we consider the paired relationships between two elements and leverage the context of a given protein relative to all others.…”

Section: Related Work and Previous Approaches To Local Threshold Detementioning

confidence: 99%

Reciprocal Perspective for Improved Protein-Protein Interaction Prediction

Dick

¹

,

Green

²

2018

Sci Rep

View full text Add to dashboard Cite

All protein-protein interaction (PPI) predictors require the determination of an operational decision threshold when differentiating positive PPIs from negatives. Historically, a single global threshold, typically optimized via cross-validation testing, is applied to all protein pairs. However, we here use data visualization techniques to show that no single decision threshold is suitable for all protein pairs, given the inherent diversity of protein interaction profiles. The recent development of high throughput PPI predictors has enabled the comprehensive scoring of all possible protein-protein pairs. This, in turn, has given rise to context, enabling us now to evaluate a PPI within the context of all possible predictions. Leveraging this context, we introduce a novel modeling framework called Reciprocal Perspective (RP), which estimates a localized threshold on a per-protein basis using several rank order metrics. By considering a putative PPI from the perspective of each of the proteins within the pair, RP rescores the predicted PPI and applies a cascaded Random Forest classifier leading to improvements in recall and precision. We here validate RP using two state-of-the-art PPI predictors, the Protein-protein Interaction Prediction Engine and the Scoring PRotein INTeractions methods, over five organisms: Homo sapiens, Saccharomyces cerevisiae, Arabidopsis thaliana, Caenorhabditis elegans, and Mus musculus. Results demonstrate the application of a post hoc RP rescoring layer significantly improves classification (p < 0.001) in all cases over all organisms and this new rescoring approach can apply to any PPI prediction method.

show abstract

“…If it is not useful to test the joint studywise hypothesis, then researchers should consider lower-order families of hypotheses and/or individual hypotheses for testing (Benjamini & Bogomolov, 2011;Efron, 2008;Fisher, 1971, p. 206;Hochberg & Tamrane, 1987, pp. 6-7;Hung & Wang, 2010;Mei et al, 2017;Rubin, 2017b). For example, in their discussion of multiple testing in microarray gene expression analysis, Yekutieli et al (2006) explained that "the set of hypotheses that is of interest to the researcher in a single study does not necessarily form a single family of hypotheses" (p. 416).…”

Section: Studywise Error Ratesmentioning

confidence: 99%

When to adjust alpha during multiple testing: A consideration of disjunction, conjunction, and individual testing.

Rubin¹

2021

Preprint

View full text Add to dashboard Cite

Scientists often adjust their significance threshold (alpha level) during null hypothesis significance testing in order to take into account multiple testing and multiple comparisons. This alpha adjustment has become particularly relevant in the context of the replication crisis in science. The present article considers the conditions in which this alpha adjustment is appropriate and the conditions in which it is inappropriate. A distinction is drawn between three types of multiple testing: disjunction testing, conjunction testing, and individual testing. It is argued that alpha adjustment is only appropriate in the case of disjunction testing, in which at least one test result must be significant in order to reject the associated joint null hypothesis. Alpha adjustment is inappropriate in the case of conjunction testing, in which all relevant results must be significant in order to reject the joint null hypothesis. Alpha adjustment is also inappropriate in the case of individual testing, in which each individual result must be significant in order to reject each associated individual null hypothesis. The conditions under which each of these three types of multiple testing is warranted are examined. It is concluded that researchers should not automatically (mindlessly) assume that alpha adjustment is necessary during multiple testing. Illustrations are provided in relation to joint studywise hypotheses and joint multiway ANOVAwise hypotheses.

show abstract

The performance of a new local false discovery rate method on tests of association between coronary artery disease (CAD) and genome-wide genetic variants

Cited by 6 publications

References 35 publications

When to adjust alpha during multiple testing: a consideration of disjunction, conjunction, and individual testing

When to adjust alpha during multiple testing: a consideration of disjunction, conjunction, and individual testing

Reciprocal Perspective for Improved Protein-Protein Interaction Prediction

When to adjust alpha during multiple testing: A consideration of disjunction, conjunction, and individual testing.

Contact Info

Product

Resources

About