High-dimensional principal component analysis with heterogeneous missingness

Abstract

We study the problem of high-dimensional Principal Component Analysis (PCA) with missing observations. In simple, homogeneous missingness settings with a noise level of constant order, we show that an existing inverse-probability weighted (IPW) estimator of the leading principal components can (nearly) attain the minimax optimal rate of convergence. However, deeper investigation reveals both that, particularly in more realistic settings where the missingness mechanism is heterogeneous, the empirical performance of the IPW estimator can be unsatisfactory, and moreover that, in the noiseless case, it fails to provide exact recovery of the principal components. Our main contribution, then, is to introduce a new method for high-dimensional PCA, called primePCA, that is designed to cope with situations where observations may be missing in a heterogeneous manner. Starting from the IPW estimator, primePCA iteratively projects the observed entries of the data matrix onto the column space of our current estimate to impute the missing entries, and then updates our estimate by computing the leading right singular space of the imputed data matrix. It turns out that the interaction between the heterogeneity of missingness and the low-dimensional structure is crucial in determining the feasibility of the problem. We therefore introduce an incoherence condition on the principal components and prove that in the noiseless case, the error of primePCA converges to zero at a geometric rate when the signal strength is not too small. An important feature of our theoretical guarantees is that they depend on average, as opposed to worst-case, properties of the missingness mechanism. Our numerical studies on both simulated and real data reveal that primePCA exhibits very encouraging performance across a wide range of scenarios.

Causal Matrix Completion

“…This is a generalization of the MAR setting as such an approach circumvents the requirement of meaningful auxiliary features X to conduct propensity score estimation. Additional works within the MNAR literature include Zhu et al (2019); Sportisse et al (2020a,b); Wang et al (2020).…”

“…That is, the entries are missing not at random (MNAR). To address the above challenges, there has been exciting recent progress on matrix completion with MNAR data, including Schnabel et al (2016); Ma and Chen (2019); Zhu et al (2019); Sportisse et al (2020a,b); Wang et al (2020); Yang et al (2021); Bhattacharya and Chatterjee (2021). Through numerous empirical studies, these works have shown that algorithms that account for MNAR data outperform conventional algorithms that are designed for MCAR data.…”

Inference for Heteroskedastic PCA with Missing Data

“…Inadequacy of prior works. While methods for estimating principal subspace are certainly not in shortage (e.g., Balzano et al (2018); Cai et al (2021); Cai and Zhang (2018); Li et al (2021); Lounici (2014); Zhang et al (2018); Zhu et al (2019)), methods for constructing confidence regions for principal subspace remain vastly under-explored. The fact that the estimators in use for PCA are typically nonlinear and nonconvex presents a substantial challenge in the development of a distributional theory, let alone uncertainty quantification.…”

Minimax Estimation of Linear Functions of Eigenvectors in the Face of Small Eigen-Gaps

“…Several useful extensions have been developed tailored to high-dimensional statistical applications, particularly when the perturbation matrix of interest enjoys certain random structure , O'Rourke et al, 2018, Vu, 2011, Wang, 2015, Xia, 2019, Yu et al, 2015. In particular, the ℓ 2 perturbation bounds for the eigenvector (or eigenspace) of the sample covariance matrix has been extensively studied in the PCA literature, e.g., [Johnstone and Lu, 2009, Lounici, 2013, 2014, Nadler, 2008, Zhu et al, 2019. Another line of works [O'Rourke et al, 2018, Vu, 2011 improved Davis-Kahan's and Wedin's theorems in the matrix denoising setting with small eigengaps, which, however, is not tight unless the spectral norm H of the noise matrix is extremely small.…”