Robust covariance estimation for high‐dimensional compositional data with application to microbial communities analysis

Abstract

Microbial communities analysis is drawing growing attention due to the rapid development fire of high‐throughput sequencing techniques nowadays. The observed data has the following typical characteristics: it is high‐dimensional, compositional (lying in a simplex) and even would be leptokurtic and highly skewed due to the existence of overly abundant taxa, which makes the conventional correlation analysis infeasible to study the co‐occurrence and co‐exclusion relationship between microbial taxa. In this article, we address the challenges of covariance estimation for this kind of data. Assuming the basis covariance matrix lying in a well‐recognized class of sparse covariance matrices, we adopt a proxy matrix known as centered log‐ratio covariance matrix in the literature. We construct a Median‐of‐Means estimator for the centered log‐ratio covariance matrix and propose a thresholding procedure that is adaptive to the variability of individual entries. By imposing a much weaker finite fourth moment condition compared with the sub‐Gaussianity condition in the literature, we derive the optimal rate of convergence under the spectral norm. In addition, we also provide theoretical guarantee on support recovery. The adaptive thresholding procedure of the MOM estimator is easy to implement and gains robustness when outliers or heavy‐tailedness exist. Thorough simulation studies are conducted to show the advantages of the proposed procedure over some state‐of‐the‐arts methods. At last, we apply the proposed method to analyze a microbiome dataset in human gut.

A new robust covariance matrix estimation for high‐dimensional microbiome data

“…Assumption 1 guarantees sparsity of the basis covariance matrix Σ and the scaling condition. Assumption 2 claims that Y j has uniformly bounded 2 + moments, which is weaker than the corresponding assumptions in Cao, Lin & Li (2019), He, Liu, Zhang & Zhou (2021), or Li, Srinivasan, Chen & Xue (2023). Furthermore, combining some simple algebraic operations and the Hölder inequality gives that max 1≤j ≤p E(|Z j | 2+ ) ≤ 8κ 2 .…”

“…Assumptions 1 and 2 are common in the covariance matrix estimation literature such as Cao, Lin & Li (2019) and He, Liu, Zhang & Zhou (2021). Assumption 1 guarantees sparsity of the basis covariance matrix $$$ \boldsymbol{\Sigma} $$$ and the scaling condition.…”

“…But their method still belongs to the category of regularising sample covariance matrix and thus is sensitive to data, violating the ideal sub‐Gaussianity assumption. To improve the robustness, He, Liu, Zhang & Zhou (2021) proposed a robust covariance estimation for compositional data (RCEC). They first construct a median of means (MoM) estimator for a proxy matrix, which is almost indistinguishable from the real basis covariance matrix.…”

“…We obtain the final estimator $$$ \hat{\boldsymbol{\Sigma}} $$$ by thresholding $$$ \hat{\boldsymbol{\Gamma}} $$$. Some existing methods such as Cao, Lin & Li (2019), He, Liu, Zhang & Zhou (2021), and Li, Srinivasan, Chen & Xue (2023) can be incorporated into our framework. In particular, this paper uses a Huber‐type estimator to estimate the proxy matrix under the assumption of finite $$$ 2+\epsilon $$$ moments for some $$$ \epsilon \in \left(0,2\right] $$$.…”

Critical review of 16S rRNA gene sequencing workflow in microbiome studies: From primer selection to advanced data analysis

“…As an alternative to avoid the ALR drawbacks, the centre of mass of all taxa can be used as a reference. Thus, the transformation known as centred log‐ratio (CLR) calculates, within each sample and for each taxon, the log‐ratios relative to the geometric mean of each taxon (He et al., 2021; Lin & Peddada, 2020b). In this case, geometric mean entangles all components of a composition in each CLR coordinate, which hampers interpretation (Gordon‐Rodríguez, 2022); although the transformation is an isometry, the sum of the transformed values equals zero, leading to a degenerate distribution (Lin & Peddada, 2020b).…”