Shrinkage improves estimation of microbial associations under different normalization methods

Abstract

Estimation of statistical associations in microbial genomic survey count data is fundamental to microbiome research. Experimental limitations, including count compositionality, low sample sizes and technical variability, obstruct standard application of association measures and require data normalization prior to statistical estimation. Here, we investigate the interplay between data normalization, microbial association estimation and available sample size by leveraging the large-scale American Gut Project (AGP) survey data. We analyze the statistical properties of two prominent linear association estimators, correlation and proportionality, under different sample scenarios and data normalization schemes, including RNA-seq analysis workflows and log-ratio transformations. We show that shrinkage estimation, a standard statistical regularization technique, can universally improve the quality of taxon–taxon association estimates for microbiome data. We find that large-scale association patterns in the AGP data can be grouped into five normalization-dependent classes. Using microbial association network construction and clustering as downstream data analysis examples, we show that variance-stabilizing and log-ratio approaches enable the most taxonomically and structurally coherent estimates. Taken together, the findings from our reproducible analysis workflow have important implications for microbiome studies in multiple stages of analysis, particularly when only small sample sizes are available.

Bacterial low-abundant taxa are key determinants of a healthy airway metagenome in the early years of human life

“…Correspondingly, investigations of the functional capacities of the core and rare species biosphere are important to gain insights into the more stable part of community airway ecology in infancy. We thus investigated the functional capacity of core and rare background species based on the reference genomes for which uniform read distributions were obtained with raspir [30] . The tool gapseq utilised a pre-defined protein sequence reference pool for every sub-reaction involved in known core metabolic pathways of bacteria (n = 1779) [34] .…”

“…Our recent software publication raspir in combination with gapseq , a tool introduced by Zimmermann et al (2021) facilitated the taxonomic and functional identification of core and rare species from shotgun metagenomic sequencing data and reference genomes, respectively, with reduced false discovery and omission rates [27] , [28] . Since previous reports have demonstrated that metagenome investigations are affected by the reference database of choice [29] and the normalisation strategy of count data for addressing the compositional behaviour of microbiome sequencing data [30] , [31] , [32] , [33] , we tested our model simulations, random forest bootstrapping aggregations, ecological network analysis and kernel-based machine learning applications on infant metagenome datasets, generated from read alignments towards either a pan-genome or a one-strain-per-species reference database. Moreover, we generated datasets based on three different read count normalisation strategies, namely variance-stabilising transformations (VST), relative log expression (RLE) and bacterial to human cell ratios (BCPHC) and worked with three distinct rarity thresholds (15th, 25th and 35th species abundance percentile) to define the core and rare species biosphere.…”

NetCoMi: network construction and comparison for microbiome data in R

“…Normalization techniques are required to make read counts comparable across different samples [ 47 , 48 ] (step 1c in Figure 1 ). The normalization approaches included in are summarized in Table 2 .…”

“…The normalization approaches included in are summarized in Table 2 . A description of these methods is available in [ 47 , 48 ]. Note that forcing the read counts of each sample to a unique sum (as done with total sum scaling) does not change the compositional structure.…”

“…Aitchison [ 45 ] suggested using the centered log-ratio (clr) transformation to move compositional data from the simplex to real space. Badri et al [ 47 ] have shown that variance-stabilizing transformations (VSTs) [ 49 ] as well as the clr-transformation produce very similar Pearson correlation estimates, which are more consistent across different sample sizes than TSS, CSS and COM methods.…”