Statistical Inferences of Linear Forms for Noisy Matrix Completion

Dong, Xueyi; Yuan, Ming

doi:10.1111/rssb.12400

Cited by 23 publications

(12 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Remark 4 (Proof sketch of Theorem 6). The proof scheme for Theorem 6 is essentially different from many recent literature on the entrywise inference (Chen et al, 2019b;Xia and Yuan, 2020;Cai et al, 2020) and we provide a proof sketch here. Without loss generality, we assume σ = 1 and u, û , v, v , w, ŵ ≥ 0.…”

Section: Entry-wise Inference For Rank-1 Tensorsmentioning

confidence: 98%

Inference for Low-rank Tensors -- No Need to Debias

Dong¹,

Zhang²,

Zhou³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper, we consider the statistical inference for several low-rank tensor models. Specifically, in the Tucker low-rank tensor PCA or regression model, provided with any estimates achieving some attainable error rate, we develop the data-driven confidence regions for the singular subspace of the parameter tensor based on the asymptotic distribution of an updated estimate by two-iteration alternating minimization. The asymptotic distributions are established under some essential conditions on the signal-to-noise ratio (in PCA model) or sample size (in regression model). If the parameter tensor is further orthogonally decomposable, we develop the methods and theory for inference on each individual singular vector. For the rank-one tensor PCA model, we establish the asymptotic distribution for general linear forms of principal components and confidence interval for each entry of the parameter tensor. Finally, numerical simulations are presented to corroborate our theoretical discoveries.In all these models, we observe that different from many matrix/vector settings in existing work, debiasing is not required to establish the asymptotic distribution of estimates or to make statistical inference on low-rank tensors. In fact, due to the widely observed statisticalcomputational-gap for low-rank tensor estimation, one usually requires stronger conditions than the statistical (or information-theoretic) limit to ensure the computationally feasible estimation is achievable. Surprisingly, such conditions "incidentally" render a feasible low-rank tensor inference without debiasing. IntroductionAn mth order tensor is a multiway array along m directions. Recent years have witnessed a fast growing demand for the collection, processing, and analysis of data in the form of tensors. These tensor data commonly arise, to name a few, when features are collected from different domains, or

show abstract

Section: Entry-wise Inference For Rank-1 Tensorsmentioning

confidence: 98%

Inference for Low-rank Tensors -- No Need to Debias

Dong¹,

Zhang²,

Zhou³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…This is in stark constrat to sparse estimation and learning problems, for which the construction of confidence regions has been extensively studied [54]- [64]. A few exceptions are worth mentioning: (1) [65]- [67] identified 2 confidence regions that are likely to cover the low-rank matrix of interest, which, however, might be loose in terms of the pre-constant; (2) focusing on lowrank matrix completion, the recent work [68] developed a debiasing strategy that constructs both confidence regions for low-rank factors and entrywise confidence intervals for the unknown matrix, attaining statistical optimality in terms of both the pre-constant and the rate; an independent work by Xia et al [69] analyzed a similar de-biasing strategy with the aid of double sample splitting, and shows asymptotic normality of linear forms of the matrix estimator; (3) [70], [71] developed a spectral projector to construct confidence regions for singular subspaces in the presence of i.i.d. additive noise; (4) [18] considered estimating linear forms of eigenvectors in a different covariance estimation model, whose analysis relies on the Gaussianity assumption; (5) [72] characterized the asymptotic normality of bilinear forms of eigenvectors, which accommodates heterogeneous noise; and (6) [44] established the 2,∞ distributional guarantees for two spectral estimators (i.e., plain SVD and heteroskedastic PCA) tailored to PCA with heteroskedastic and missing data.…”

Section: Prior Artmentioning

confidence: 99%

Tackling Small Eigen-Gaps: Fine-Grained Eigenvector Estimation and Inference Under Heteroscedastic Noise

Chen

Wei

Chen

2021

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

This paper aims to address two fundamental challenges arising in eigenvector estimation and inference for a low-rank matrix from noisy observations: (1) how to estimate an unknown eigenvector when the eigen-gap (i.e. the spacing between the associated eigenvalue and the rest of the spectrum) is particularly small; (2) how to perform estimation and inference on linear functionals of an eigenvector -a sort of "fine-grained" statistical reasoning that goes far beyond the usual 2 analysis.We investigate how to address these challenges in a setting where the unknown n × n matrix is symmetric and the additive noise matrix contains independent (and non-symmetric) entries. Based on eigen-decomposition of the asymmetric data matrix, we propose estimation and uncertainty quantification procedures for an unknown eigenvector, which further allow us to reason about linear functionals of an unknown eigenvector. The proposed procedures and the accompanying theory enjoy several important features: (1) distribution-free (i.e. prior knowledge about the noise distributions is not needed); (2) adaptive to heteroscedastic noise; (3) minimax optimal under Gaussian noise. Along the way, we establish valid procedures to construct confidence intervals for the unknown eigenvalues. All this is guaranteed even in the presence of a small eigen-gap (up to O( n/poly log(n) ) times smaller than the requirement in prior theory), which goes significantly beyond what generic matrix perturbation theory has to offer.

show abstract

“…Spectral methods have been successfully applied to tackle noisy matrix completion (Chen et al, 2020a;Chen and Wainwright, 2015;Cho et al, 2017;Keshavan et al, 2010a;Ma et al, 2020;Sun and Luo, 2016;Zheng and Lafferty, 2016), which commonly serve as an effective initialization scheme for nonconvex optimization methods (Chi et al, 2019). While statistical inference for noisy matrix completion has been investigated recently (Chen et al, 2019c;Chernozhukov et al, 2021;Xia and Yuan, 2021), these prior works focused on performing inference based on optimization-based estimators. How to construct fine-grained confidence intervals based on spectral methods remains previously out of reach for noisy matrix completion.…”

Section: Other Related Workmentioning

confidence: 99%

Inference for Heteroskedastic PCA with Missing Data

Yan,

Chen,

Fan

2021

Preprint

View full text Add to dashboard Cite

This paper studies how to construct confidence regions for principal component analysis (PCA) in high dimension, a problem that has been vastly under-explored. While computing measures of uncertainty for nonlinear/nonconvex estimators is in general difficult in high dimension, the challenge is further compounded by the prevalent presence of missing data and heteroskedastic noise. We propose a suite of solutions to perform valid inference on the principal subspace based on two estimators: a vanilla SVD-based approach, and a more refined iterative scheme called HeteroPCA (Zhang et al., 2018). We develop non-asymptotic distributional guarantees for both estimators, and demonstrate how these can be invoked to compute both confidence regions for the principal subspace and entrywise confidence intervals for the spiked covariance matrix. Particularly worth highlighting is the inference procedure built on top of HeteroPCA, which is not only valid but also statistically efficient for broader scenarios (e.g., it covers a wider range of missing rates and signal-to-noise ratios). Our solutions are fully data-driven and adaptive to heteroskedastic random noise, without requiring prior knowledge about the noise levels and noise distributions.

show abstract

Statistical Inferences of Linear Forms for Noisy Matrix Completion

Cited by 23 publications

References 43 publications

Inference for Low-rank Tensors -- No Need to Debias

Inference for Low-rank Tensors -- No Need to Debias

Tackling Small Eigen-Gaps: Fine-Grained Eigenvector Estimation and Inference Under Heteroscedastic Noise

Inference for Heteroskedastic PCA with Missing Data

Contact Info

Product

Resources

About