A broken promise: microbiome differential abundance methods do not control the false discovery rate

Hawinkel, Stijn; Mattiello, Federico; Bijnens, Luc; Thas, Olivier

doi:10.1093/bib/bbx104

Cited by 168 publications

(265 citation statements)

References 59 publications

Supporting

Mentioning

229

Contrasting

Unclassified

Order By: Relevance

“…While ALDEx2 controlled FDR under S1 and S2, its sensitivity is consistently lower than the LDM-based methods. We have also included edgeR in this study, whose highly inflated FDR corroborated the finding in Hawinkel et al [2017].…”

Section: Results For Testing Individual Otus With Independent Samplessupporting

confidence: 75%

“…We also provide a new version of the PERMANOVA test based on our approach that we show outperforms the functions adonis and adonis2 in the R package vegan, the most commonly used implementations of PERMANOVA. Recent simulation studies suggest that many microbiome analysis methods fail to control the FDR when applied to overdispersed data [Hawinkel et al, 2017]. We show that the LDM controls FDR in exactly the kind of situations where other methods fail.…”

Section: Introductionmentioning

confidence: 79%

“…We also calculated OTU-specific tests using the LDM, which we compared to results from DESeq2. We only performed limited comparisons to aMiSPU, edgeR, MetagenomeSeq, Wilcoxon rank-sum test (applied to OTU frequencies), ANCOM, and ALDEx2, because aMiSPU does not allow for clustered data, edgeR generally has highly inflated FDR [Hawinkel et al, 2017], and the others do not allow for confounding covariates.…”

Section: Simulation Studiesmentioning

confidence: 99%

See 2 more Smart Citations

Testing hypotheses about the microbiome using the linear decomposition model (LDM)

Satten²

2017

Preprint

View full text Add to dashboard Cite

Motivation: Methods for analyzing microbiome data generally fall into one of two groups: tests of the global hypothesis of any microbiome effect, which do not provide any information on the contribution of individual operational taxonomic units (OTUs); and tests for individual OTUs, which do not typically provide a global test of microbiome effect. Without a unified approach, the findings of a global test may be hard to resolve with the findings at the individual OTU level. Further, many tests of individual OTU effects do not preserve the false discovery rate (FDR). Results:We introduce the linear decomposition model (LDM), that provides a single analysis path that includes global tests of any effect of the microbiome, tests of the effects of individual OTUs while accounting for multiple testing by controlling the FDR, and a connection to distance-based ordination. The LDM accommodates both continuous and discrete variables (e.g., clinical outcomes, environmental factors) as well as interaction terms to be tested either singly or in combination, allows for adjustment of confounding covariates, and uses permutation-based p-values that can control for correlation. The LDM can also be applied to transformed data, and an "omnibus" test can easily combine results from analyses conducted on different transformation scales. We also provide a new implementation of PERMANOVA based on our approach. For global testing, our simulations indicate the LDM provided correct type I error and can have comparable power to existing distance-based methods. For testing individual OTUs, our simulations indicate the LDM controlled the FDR well. In contrast, DESeq2 often had inflated FDR; MetagenomeSeq generally had the lowest sensitivity. The flexibility of the LDM for a variety of microbiome studies is illustrated by the analysis of data from two microbiome studies. We also show that our implementation of PERMANOVA can outperform existing implementations. Availability and Implementation:The R package LDM is available on GitHub at https://github.com/yijuanhu/LDM in formats appropriate for Macintosh or Windows.

show abstract

Section: Results For Testing Individual Otus With Independent Samplessupporting

confidence: 75%

Section: Introductionmentioning

confidence: 79%

Section: Simulation Studiesmentioning

confidence: 99%

See 1 more Smart Citation

Testing hypotheses about the microbiome using the linear decomposition model (LDM)

Satten²

2017

Preprint

View full text Add to dashboard Cite

show abstract

“…Second, it fails to account for the simplex nature of compositional data and may suffer from spurious negative correlations imposed by the fact that relative abundances across all taxa must sum to one within a given microbiome sample. As a consequence, traditional FDR control procedures (Benjamini and Hochberg, 1995) may not work for microbiome-wide multiple testing (Hawinkel et al, 2017).…”

Section: Introductionmentioning

confidence: 99%

“…Without FDR control, existing joint microbiome fine-mapping methods can produce less reliable discoveries and would probably lead to costly and fruitless downstream validation and functional studies (Wang and Jia, 2016;Hawinkel et al, 2017).…”

Section: Introductionmentioning

confidence: 99%

Compositional knockoff filter for high-dimensional regression analysis of microbiome data

Srinivasan

Zhan

Xue

2019

Preprint

View full text Add to dashboard Cite

A critical task in microbiome data analysis is to explore the association between a scalar response of interest and a large number of microbial taxa that are summarized as compositional data at different taxonomic levels. Motivated by fine-mapping of the microbiome, we propose a two-step compositional knockoff filter (CKF) to provide the effective finite-sample false discovery rate (FDR) control in high-dimensional linear log-contrast regression analysis of microbiome compositional data. In the first step, we employ the compositional screening procedure to remove insignificant microbial taxa while retaining the essential sum-to-zero constraint. In the second step, we extend the knockoff filter to identify the significant microbial taxa in the sparse regression model for compositional data.Thereby, a subset of the microbes is selected from the high-dimensional microbial taxa as related to the response using a pre-specified FDR threshold. We study the asymptotic properties of the proposed two-step procedure, including both sure screening and effective false discovery control. We demonstrate the finite-sample properties in simulation studies, which show the gain in the empirical power while controlling the nominal FDR. The potential usefulness of the proposed method is also illustrated with application to an inflammatory bowel disease dataset to identify microbial taxa that influence host gene expressions.

show abstract

QIIME 2 Enables Comprehensive End‐to‐End Analysis of Diverse Microbiome Data and Comparative Studies with Publicly Available Data

Estaki

Jiang

Bokulich

et al. 2020

CP in Bioinformatics

277

190

View full text Add to dashboard Cite

QIIME 2 is a completely re‐engineered microbiome bioinformatics platform based on the popular QIIME platform, which it has replaced. QIIME 2 facilitates comprehensive and fully reproducible microbiome data science, improving accessibility to diverse users by adding multiple user interfaces. QIIME 2 can be combined with Qiita, an open‐source web‐based platform, to re‐use available data for meta‐analysis. The following basic protocol describes how to install QIIME 2 on a single computer and analyze microbiome sequence data, from processing of raw DNA sequence reads through generating publishable interactive figures. These interactive figures allow readers of a study to interact with data with the same ease as its authors, advancing microbiome science transparency and reproducibility. We also show how plug‐ins developed by the community to add analysis capabilities can be installed and used with QIIME 2, enhancing various aspects of microbiome analyses—e.g., improving taxonomic classification accuracy. Finally, we illustrate how users can perform meta‐analyses combining different datasets using readily available public data through Qiita. In this tutorial, we analyze a subset of the Early Childhood Antibiotics and the Microbiome (ECAM) study, which tracked the microbiome composition and development of 43 infants in the United States from birth to 2 years of age, identifying microbiome associations with antibiotic exposure, delivery mode, and diet. For more information about QIIME 2, see https://qiime2.org. To troubleshoot or ask questions about QIIME 2 and microbiome analysis, join the active community at https://forum.qiime2.org. © 2020 The Authors. Basic Protocol: Using QIIME 2 with microbiome data Support Protocol: Further microbiome analyses

show abstract

A broken promise: microbiome differential abundance methods do not control the false discovery rate

Cited by 168 publications

References 59 publications

Testing hypotheses about the microbiome using the linear decomposition model (LDM)

Testing hypotheses about the microbiome using the linear decomposition model (LDM)

Compositional knockoff filter for high-dimensional regression analysis of microbiome data

QIIME 2 Enables Comprehensive End‐to‐End Analysis of Diverse Microbiome Data and Comparative Studies with Publicly Available Data

Contact Info

Product

Resources

About