Stochastic variational variable selection for high-dimensional microbiome data

Abstract

Background The rapid and accurate identification of a minimal-size core set of representative microbial species plays an important role in the clustering of microbial community data and interpretation of clustering results. However, the huge dimensionality of microbial metagenomics datasets is a major challenge for the existing methods such as Dirichlet multinomial mixture (DMM) models. In the approach of the existing methods, the computational burden of identifying a small number of representative species from a large number of observed species remains a challenge. Results We propose a novel approach to improve the performance of the widely used DMM approach by combining three ideas: (i) we propose an indicator variable to identify representative operational taxonomic units that substantially contribute to the differentiation among clusters; (ii) to address the computational burden of high-dimensional microbiome data, we propose a stochastic variational inference, which approximates the posterior distribution using a controllable distribution called variational distribution, and stochastic optimization algorithms for fast computation; and (iii) we extend the finite DMM model to an infinite case by considering Dirichlet process mixtures and estimating the number of clusters as a variational parameter. Using the proposed method, stochastic variational variable selection (SVVS), we analyzed the root microbiome data collected in our soybean field experiment, the human gut microbiome data from three published datasets of large-scale case-control studies and the healthy human microbiome data from the Human Microbiome Project. Conclusions SVVS demonstrates a better performance and significantly faster computation than those of the existing methods in all cases of testing datasets. In particular, SVVS is the only method that can analyze massive high-dimensional microbial data with more than 50,000 microbial species and 1000 samples. Furthermore, a core set of representative microbial species is identified using SVVS that can improve the interpretability of Bayesian mixture models for a wide range of microbiome studies.

Critical review of 16S rRNA gene sequencing workflow in microbiome studies: From primer selection to advanced data analysis

“…As with generic ones, there are options available for working in the R environment (R Core Team), such as Dirichlet Multinomial Mixtures (DMM) (Morgan, 2023) or coda4microbiome (Calle et al, 2023), in the Python environment (Python Software Foundation), such as TA B L E 6 Distinct methods for variable selection designed for microbiome data, including their main advantages and disadvantages. Source: The information for the construction of this table was taken from Dang et al (2022), Hinton and Mucha (2022), Koh and Zhao (2020), Rivera-Pinto et al (2018), Wilson et al (2021) and Wu et al (2016).…”

“… Source : The information for the construction of this table was taken from Dang et al. (2022), Hinton and Mucha (2022), Koh and Zhao (2020), Rivera‐Pinto et al. (2018), Wilson et al.…”

oFVSD: a Python package of optimized forward variable selection decoder for high-dimensional neuroimaging data

“…It is also one of the first and most popular algorithms for causal feature selection in some fields, such as gene selection, microarray data analysis, and gene expression data analysis ( Ooi and Tan, 2003 ; Blanco et al, 2004 ; Jirapech-Umpai and Aitken, 2005 ; Saeys et al, 2007 ). The powerful nature of feature decoding in the analysis of high-dimensional microbiome data has also been demonstrated ( Dang et al, 2022 ; Dang and Kishino, 2022 ). The FVS can be a powerful additional tool for neuroimaging research.…”

An integrative framework of stochastic variational variable selection for joint analysis of multi-omics microbiome data

“…For example, given the complex nature of metagenomic data, the current BCC and Clusternomics approaches cannot cluster communities into groups with similar compositions. In contrast, the Dirichlet multinomial mixture (DMM) [20,21] is a successful method for probabilistic modeling of microbial metagenomics data. In a previous study [21],…”

“…In contrast, the Dirichlet multinomial mixture (DMM) [20,21] is a successful method for probabilistic modeling of microbial metagenomics data. In a previous study [21],…”