Asymmetry helps: Eigenvalue and eigenvector analyses of asymmetrically perturbed low-rank matrices

Abstract

This paper is concerned with the interplay between statistical asymmetry and spectral methods. Suppose we are interested in estimating a rank-1 and symmetric matrix M ∈ R n×n , yet only a randomly perturbed version M is observed. The noise matrix M − M is composed of independent (but not necessarily homoscedastic) entries and is, therefore, not symmetric in general. This might arise if, for example, when we have two independent samples for each entry of M and arrange them in an asymmetric fashion. The aim is to estimate the leading eigenvalue and the leading eigenvector of M .We demonstrate that the leading eigenvalue of the data matrix M can be O( √ n) times more accurate (up to some log factor) than its (unadjusted) leading singular value of M in eigenvalue estimation. Moreover, the eigendecomposition approach is fully adaptive to heteroscedasticity of noise, without the need of any prior knowledge about the noise distributions. In a nutshell, this curious phenomenon arises since the statistical asymmetry automatically mitigates the bias of the eigenvalue approach, thus eliminating the need of careful bias correction. Additionally, we develop appealing nonasymptotic eigenvector perturbation bounds; in particular, we are able to bound the perturbation of any linear function of the leading eigenvector of M (e.g., entrywise eigenvector perturbation). We also provide partial theory for the more general rank-r case. The takeaway message is this: arranging the data samples in an asymmetric manner and performing eigendecomposition could sometimes be quite beneficial.

Subspace estimation from unbalanced and incomplete data matrices: ℓ2,∞ statistical guarantees

“…Further, entrywise statistical analysis has recently received significant attention for various statistical problems [4,5,23,24,27,30,42,45,74,82,90,97,108,115]. For instance, entrywise guarantees for spectral methods are obtained in [27,42] based on an algebraic Neumann trick, while the results in [4,30,115] were established based on a leave-one-out analysis. The work by [27,69,70] went one step further by controlling an arbitrary linear form of the eigenvectors or singular vectors of interest.…”

“…For instance, entrywise guarantees for spectral methods are obtained in [27,42] based on an algebraic Neumann trick, while the results in [4,30,115] were established based on a leave-one-out analysis. The work by [27,69,70] went one step further by controlling an arbitrary linear form of the eigenvectors or singular vectors of interest. These results, however, typically lead to suboptimal performance guarantees when the row dimension and the column dimension of the matrix are substantially different.…”

“…It is worth noting that low-rank subspace estimation from noisy and incomplete data has been extensively studied in a large number of prior work (e.g., [4,27,35,38,79,113,117]). While many of these prior results allow d 1 and d 2 to differ, they typically fall short of establishing optimal dependency on d 1 and d 2 in the highly unbalanced scenarios.…”

Tackling Small Eigen-Gaps: Fine-Grained Eigenvector Estimation and Inference Under Heteroscedastic Noise

“…A glimpse of our approach and our contributions a) Eigen-decomposition meets statistical asymmetry: The current paper makes progress in a setting where the noise matrix H consists of independent (but possibly heterogeneous and heteroscedastic) zero-mean components. Our approach is inspired by the findings of [17]. Consider, for example, the case when H is a random and asymmetric matrix and when M is a rank-1 symmetric matrix.…”

Normal approximation and confidence region of singular subspaces

“…As a result, a much larger than regular sample size requirement is necessary to tradeoff the estimation error of the expected projection distance. Most recently, Chen et al (2018) revealed an interesting phenomenon of the perturbation of eigenvalues and eigenvectors of such non-asymmetric random perturbations, showing that the perturbation of eigen structures is much smaller than the singular structures. In addition, some non-asymptotic perturbation bounds of empirical singular vectors can be found in Koltchinskii and Xia (2016), Bloemendal et al (2016) and Abbe et al (2017).…”