Exploiting the Structure Effectively and Efficiently in Low-Rank Matrix Recovery

Abstract

Low rank model arises from a wide range of applications, including machine learning, signal processing, computer algebra, computer vision, and imaging science. Low rank matrix recovery is about reconstructing a low rank matrix from incomplete measurements. In this survey we review recent developments on low rank matrix recovery, focusing on three typical scenarios: matrix sensing, matrix completion and phase retrieval. An overview of effective and efficient approaches for the problem is given, including nuclear norm minimization, projected gradient descent based on matrix factorization, and Riemannian optimization based on the embedded manifold of low rank matrices. Numerical recipes of different approaches are emphasized while accompanied by the corresponding theoretical recovery guarantees. This is an ill-posed problem without assuming any structure on X since there are more unknowns than equations. However, noticing that the number of degrees of freedom in an n by n rank r matrix is p2n´rqr [100] which can be much smaller than n 2 provided r is small, it is reasonable to expect to reconstruct a low rank matrix from fewer than n 2 measurements.Authors are listed alphabetically.

Toward the Optimal Construction of a Loss Function Without Spurious Local Minima for Solving Quadratic Equations

“…For more general low rank matrix recovery, a variety of nonconvex algorithms have been developed and analyzed, including those based on matrix factorization [38,50] and those based on the embedded manifold of low rank matrices [45,44]. The reader can refer to the review paper [8] for more details. Geometric landscape of related loss functions for low rank matrix recovery has been investigated in [18,19,28,4,32].…”

Low-rank Tensor Estimation via Riemannian Gauss-Newton: Statistical Optimality and Second-Order Convergence

“…Many progresses in this topic were made for the low-rank matrix estimation (Keshavan et al, 2009;Boumal andAbsil, 2011, 2015;Wei et al, 2016;Meyer et al, 2011;Mishra et al, 2014;Vandereycken, 2013;Huang and Hand, 2018;Cherian and Sra, 2016;Luo et al, 2020). See the recent survey on this line of work at Cai and Wei (2018); Uschmajew and Vandereycken (2020). Moreover, Riemannian manifold optimization method has been applied for various problems on low-rank tensor estimation, such as tensor regression (Cai et al, 2020;Kressner et al, 2016), tensor completion (Rauhut et al, 2015;Kasai and Mishra, 2016;Dong et al, 2021;Kressner et al, 2014;Heidel and Schulz, 2018;Xia and Yuan, 2017;Steinlechner, 2016;Da Silva and Herrmann, 2015), and robust tensor PCA (Cai et al, 2021).…”

Nonconvex Factorization and Manifold Formulations are Almost Equivalent in Low-rank Matrix Optimization

“…First, from an algorithmic perspective, a number of algorithms, including the penalty approaches, gradient descent, alternating minimization, Gauss-Newton, have been developed either for solving the manifold formulation [BPS20, GS10, BA11, MMBS14, MBS11, MMBS14, Van13, HH18, LHLZ20] or the factorization formulation [CLS15, JNS13, SL15, TD21, TBS `16, WYZ12,BNZ21]. We refer readers to [CLC19,CW18a] for the recent algorithmic development under two formulations. Many algorithms developed under the manifold formulation involve Riemannian optimization techniques and can be more complex than the ones developed under the factorization formulation.…”

“…Note that (3), (4) and ( 5) are unconstrained, and thus can be tackled by running unconstrained optimization algorithms. Indeed, under proper assumptions, a number of algorithms with theoretical guarantees have been proposed for both the manifold and the factorization formulations [CLC19,CW18a]. See Section 1.2 for a review of existing results.…”