Empirical Bayes estimation of posterior probabilities of enrichment: A comparative study of five estimators of the local false discovery rate

Abstract

BackgroundIn investigating differentially expressed genes or other selected features, researchers conduct hypothesis tests to determine which biological categories, such as those of the Gene Ontology (GO), are enriched for the selected features. Multiple comparison procedures (MCPs) are commonly used to prevent excessive false positive rates. Traditional MCPs, e.g., the Bonferroni method, go to the opposite extreme: strictly controlling a family-wise error rate, resulting in excessive false negative rates. Researchers generally prefer the more balanced approach of instead controlling the false discovery rate (FDR). However, the q-values that methods of FDR control assign to biological categories tend to be too low to reliably estimate the probability that a biological category is not enriched for the preselected features. Thus, we study an application of the other estimators of that probability, which is called the local FDR (LFDR).ResultsWe considered five LFDR estimators for detecting enriched GO terms: a binomial-based estimator (BBE), a maximum likelihood estimator (MLE), a normalized MLE (NMLE), a histogram-based estimator assuming a theoretical null hypothesis (HBE), and a histogram-based estimator assuming an empirical null hypothesis (HBE-EN). Since NMLE depends not only on the data but also on the specified value of Π0, the proportion of non-enriched GO terms, it is only advantageous when either Π0 is already known with sufficient accuracy or there are data for only 1 GO term. By contrast, the other estimators work without specifying Π0 but require data for at least 2 GO terms. Our simulation studies yielded the following summaries of the relative performance of each of those four estimators. HBE and HBE-EN produced larger biases for 2, 4, 8, 32, and 100 GO terms than BBE and MLE. BBE has the lowest bias if Π0 is 1 and if the number of GO terms is between 2 and 32. The bias of MLE is no worse than that of BBE for 100 GO terms even when the ideal number of components in its underlying mixture model is unknown, but has high bias when the number of GO terms is small compared to the number of estimated parameters. For unknown values of Π0, BBE has the lowest bias for a small number of GO terms (2-32 GO terms), and MLE has the lowest bias for a medium number of GO terms (100 GO terms).ConclusionsFor enrichment detection, we recommend estimating the LFDR by MLE given at least a medium number of GO terms, by BBE given a small number of GO terms, and by NMLE given either only 1 GO term or precise knowledge of Π0.

Identification of loci governing eight agronomic traits using a GBS‐GWAS approach and validation by QTL mapping in soya bean

“…Compressed mixed linear models (CMLM) incorporating a kinship matrix (K or K*) along with P or Q were tested. The negative log(1/n) was used to establish a significance threshold Yang et al, 2013).…”

Use of the Local False Discovery Rate for Identification of Metabolic Biomarkers in Rat Urine Following Genkwa Flos-Induced Hepatotoxicity

“…The FDR is the expected ratio of the number of false positives over the total number of rejections of the null hypothesis. In 1995, Benjamini and Hochberg [24] introduced a procedure to determine the null hypothesis rejection of each hypothesis comparison. The BH method provides more statistical power than the Bonferroni method.…”

Small‐scale Inference: Empirical Bayes and Confidence Methods for as Few as a Single Comparison

“…To prevent overfitting in the form of excessive variance in the estimates, the value of K must be smaller for smaller values of N (see, for example, Yang et al ., ). For that reason, Padilla & Bickel () suggested K = 1 when N < 1000.…”

“…For example, in the hierarchical model of Section 2.1, the number K of components of the mixture model should increase with N , the number of hypotheses that can be tested on the basis of the available data, as Yang et al . () discussed at length. In this case, the frequency‐type prior distribution of each δ i is unknown except for the fact that it is a member of where D 0 denotes the Dirac measure that has its probability mass at 0, each denotes the Dirac measure that has its probability mass at δ ( k ) , and are assumed lower and upper bounds of π 0 and Ψ is a set of all values of ψ satisfying and any other restrictions on and …”