Right Singular Vector Projection Graphs: Fast High Dimensional Covariance Matrix Estimation under Latent Confounding

Abstract

Summary We consider the problem of estimating a high dimensional p×p covariance matrix Σ, given n observations of confounded data with covariance Σ+ΓnormalΓsans-serifT, where Γ is an unknown p×q matrix of latent factor loadings. We propose a simple and scalable estimator based on the projection onto the right singular vectors of the observed data matrix, which we call right singular vector projection (RSVP). Our theoretical analysis of this method reveals that, in contrast with approaches based on the removal of principal components, RSVP can cope well with settings where the smallest eigenvalue of normalΓsans-serifTΓ is relatively close to the largest eigenvalue of Σ, as well as when the eigenvalues of normalΓsans-serifTΓ are diverging fast. RSVP does not require knowledge or estimation of the number of latent factors q, but it recovers Σ only up to an unknown positive scale factor. We argue that this suffices in many applications, e.g. if an estimate of the correlation matrix is desired. We also show that, by using subsampling, we can further improve the performance of the method. We demonstrate the favourable performance of RSVP through simulation experiments and an analysis of gene expression data sets collated by the GTEX consortium.

High-dimensional regression with potential prior information on variable importance

“…We used the 'muscle-skeletal' dataset from the GTEx project 2 , which has 491 rows and 14 713 columns, as our uncorrupted dataset. We preprocessed the data as in Shah et al [2020] by regressing out measured and estimated confounders. We took as a response variable a column randomly selected (anew in each run) from the matrix, meaning that for our experiment n = 491 and p = 14 712.…”

Rejoinder: Invariance, Causality and Robustness

“…The variables A would encode the first few principal components of X, and hence A would not be exogenous: then OLS+A (γ = 0) is adjusting for these first principal components and aims to remove hidden confounding bias in estimating the causal parameter β. This common practice in applied statistics (e.g., Novembre et al, 2008) can be justified under the assumption of "dense confounding" where the hidden variables H affect most of the X components ( Ćevid, Bühlmann and Meinshausen, 2018); see also closely related work by, for example, Chandrasekaran, Parrilo and Willsky (2012), Shah et al (2018), Guo, Ćevid and Bühlmann (2020). The theoretical and methodological arguments are different since A is now a proxy for the hidden latent confounder H , very different from a valid instrument and not exogenous.…”

Deconfounding and Causal Regularisation for Stability and External Validity

“…We mention here that Shah et al (2020) provide vaguely related results on robustness for the GTEx data for another Ridge-type procedure for undirected graphical models.…”