Subspace estimation from unbalanced and incomplete data matrices: ℓ2,∞ statistical guarantees

Abstract

This paper is concerned with estimating the column space of an unknown low-rank matrix A ∈ R d 1 ×d 2 , given noisy and partial observations of its entries. There is no shortage of scenarios where the observations-while being too noisy to support faithful recovery of the entire matrix-still convey sufficient information to enable reliable estimation of the column space of interest. This is particularly evident and crucial for the highly unbalanced case where the column dimension d 2 far exceeds the row dimension d 1 , which is the focal point of the current paper.We investigate an efficient spectral method, which operates upon the sample Gram matrix with diagonal deletion. While this algorithmic idea has been studied before, we establish new statistical guarantees for this method in terms of both 2 and 2,∞ estimation accuracy, which improve upon prior results if d 2 is substantially larger than d 1 . To illustrate the effectiveness of our findings, we derive matching minimax lower bounds with respect to the noise levels, and develop consequences of our general theory for three applications of practical importance: (1) tensor completion from noisy data, (2) covariance estimation/principal component analysis with missing data and (3) community recovery in bipartite graphs. Our theory leads to improved performance guarantees for all three cases.

Asymmetry helps: Eigenvalue and eigenvector analyses of asymmetrically perturbed low-rank matrices

“…Asymmetrization for other applications. Given the abundant applications of spectral estimation, our findings are likely to be useful for other matrix eigenvalue problems and might extend to the tensor case (Cai et al (2019), Zhang and Xia (2018)). Here, we conclude the paper with an example in covariance estimation (Baik and Silverstein (2006), Fan, Wang and Zhong (2017)).…”

Nonconvex Low-Rank Tensor Completion from Noisy Data

“…i } 1≤i≤r . This strategy is partly motivated by prior approaches developed for covariance estimation with missing data (Lounici 2014, Montanari and Sun 2018, Cai et al 2021. We next provide a brief introduction.…”

“…orth , respectively. Equipped with the aforementioned notation, we can invoke (Cai et al 2021, corollary 1) to arrive at the following lemma, which upper-bounds the distance between our subspace estimate U and the ground truth U ?…”

“…Lemma 8 follows immediately from the analysis of (Cai et al 2021, lemma 4). As a remark, the construction of the leave-one-out sequences herein is slightly different from the one in (Cai et al 2021). However, it is straightforward to adapt the proof of (Cai et al 2021) to the case considered herein.…”

Convex and Nonconvex Optimization Are Both Minimax-Optimal for Noisy Blind Deconvolution Under Random Designs

“…At the technical level, the pivotal idea of our paper lies in bridging convex and nonconvex estimators, which is motivated by prior works Chen et al [2020bChen et al [ , 2019cChen et al [ , 2020c on matrix completion and robust principal component analysis. Such crucial connections have been established with the assistance of the leave-one-out analysis framework, which has already proved effective in analyzing a variety of nonconvex statistical problems [ El Karoui, 2018, Chen et al, 2019a,b, Ding and Chen, 2020, Cai et al, 2020, Dong and Shi, 2018, Cai et al, 2021a, Chen et al, 2020a, Zhong and Boumal, 2018. In this subsection, we carry out a series of numerical experiments to confirm the validity of our theory.…”