Non-convex low-rank matrix recovery with arbitrary outliers via median-truncated gradient descent

Li, Yuanxin; Chi, Yuejie; Zhang, Huishuai; Liang, Y. T.

doi:10.1093/imaiai/iaz009

Cited by 34 publications

(42 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…in the robust phase retrieval problem (89), the spectral method might not work properly even with the presence of a single outlier whose magnitude can be arbitrarily large to perturb the leading eigenvector of Y . To mitigate this issue, a median-truncation scheme was proposed in [24,88], where…”

Section: Truncated Spectral Methods For Removing Sparse Outliersmentioning

confidence: 99%

“…There are two common purposes for enforcing a truncation step: (1) to remove samples whose associated design vectors are too coherent with the current iterate [20,86,87], in order to accelerate convergence and improve sample complexity; (2) to remove samples that may be adversarial outliers, in the hope of improving robustness of the algorithm [24,64,88].…”

Section: Truncated Gradient Descentmentioning

confidence: 99%

“…Another truncation strategy to remove outliers is based on the sample median, as the median is known to be robust against arbitrary outliers [24,88]. We illustrate this median-truncation approach through an example of robust phase retrieval [24], where we assume a subset of samples in (26) is corrupted arbitrarily, with their index set denoted by S with |S| = αm.…”

Section: Truncation For Removing Sparse Outliersmentioning

confidence: 99%

“…The first order-wise optimal spectral method was proposed by Chen and Candès [20], based on the truncation idea. This method has multiple variants [87,88], and has been shown to be robust against noise. The precise asymptotic characterization of the spectral method was first obtained in [144].…”

Section: Notesmentioning

confidence: 99%

See 3 more Smart Citations

Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview

Chi

Chen

2019

IEEE Trans. Signal Process.

Self Cite

318

284

View full text Add to dashboard Cite

Substantial progress has been made recently on developing provably accurate and efficient algorithms for low-rank matrix factorization via nonconvex optimization. While conventional wisdom often takes a dim view of nonconvex optimization algorithms due to their susceptibility to spurious local minima, simple iterative methods such as gradient descent have been remarkably successful in practice. The theoretical footings, however, had been largely lacking until recently.In this tutorial-style overview, we highlight the important role of statistical models in enabling efficient nonconvex optimization with performance guarantees. We review two contrasting approaches: (1) two-stage algorithms, which consist of a tailored initialization step followed by successive refinement; and (2) global landscape analysis and initialization-free algorithms. Several canonical matrix factorization problems are discussed, including but not limited to matrix sensing, phase retrieval, matrix completion, blind deconvolution, robust principal component analysis, phase synchronization, and joint alignment. Special care is taken to illustrate the key technical insights underlying their analyses. This article serves as a testament that the integrated consideration of optimization and statistics leads to fruitful research findings.

show abstract

Section: Truncated Spectral Methods For Removing Sparse Outliersmentioning

confidence: 99%

Section: Truncated Gradient Descentmentioning

confidence: 99%

Section: Truncation For Removing Sparse Outliersmentioning

confidence: 99%

Section: Notesmentioning

confidence: 99%

See 2 more Smart Citations

Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview

Chi

Chen

2019

IEEE Trans. Signal Process.

Self Cite

318

284

View full text Add to dashboard Cite

show abstract

“…However, these methods typically require performing an SVD in each iteration for projection onto the set of low-rank matrices. Recently, a median-truncated gradient descent method has been proposed in [31] to tackle (1.2), where the gradient is modified to alleviate the effect of outliers. The median-truncated gradient descent is shown to have a local linear convergence rate [31], but such guarantee requires m nr log n measurements.…”

Section: Relatedmentioning

confidence: 99%

Nonconvex Robust Low-Rank Matrix Recovery

Li¹,

Zhu²,

So³

et al. 2020

SIAM J. Optim.

View full text Add to dashboard Cite

In this paper we study the problem of recovering a low-rank matrix from a number of random linear measurements that are corrupted by outliers taking arbitrary values. We consider a nonsmooth nonconvex formulation of the problem, in which we explicitly enforce the low-rank property of the solution by using a factored representation of the matrix variable and employ an 1loss function to robustify the solution against outliers. We show that even when a constant fraction (which can be up to almost half) of the measurements are arbitrarily corrupted, as long as certain measurement operators arising from the measurement model satisfy the so-called 1 / 2 -restricted isometry property, the ground-truth matrix can be exactly recovered from any global minimum of the resulting optimization problem. Furthermore, we show that the objective function of the optimization problem is sharp and weakly convex. Consequently, a subgradient Method (SubGM) with geometrically diminishing step sizes will converge linearly to the ground-truth matrix when suitably initialized. We demonstrate the efficacy of the SubGM for the nonconvex robust low-rank matrix recovery problem with various numerical experiments.

show abstract

Low rank matrix recovery with adversarial sparse noise*

Xu¹,

Li²,

Lin³

2022

Inverse Problems

View full text Add to dashboard Cite

Many problems in data science can be treated as recovering a low-rank matrix from a small number of random linear measurements, possibly corrupted with adversarial noise and dense noise. Recently, a bunch of theories on variants of models have been developed for different noises, but with fewer theories on the adversarial noise. In this paper, we study low-rank matrix recovery problem from linear measurements perturbed by $\ell_1$-bounded noise and sparse noise that can arbitrarily change an adversarially chosen $\omega$-fraction of the measurement vector. For Gaussian measurements with nearly optimal number of measurements, we show that the nuclear-norm constrained least absolute deviation (LAD) can successfully estimate the ground-truth matrix for any $\omega<0.239$. Similar robust recovery results are also established for an iterative hard thresholding algorithm applied to the rank-constrained LAD considering geometrically decaying step-sizes, and the unconstrained LAD based on matrix factorization as well as its subgradient descent solver.

show abstract

Non-convex low-rank matrix recovery with arbitrary outliers via median-truncated gradient descent

Cited by 34 publications

References 42 publications

Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview

Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview

Nonconvex Robust Low-Rank Matrix Recovery

Low rank matrix recovery with adversarial sparse noise*

Contact Info

Product

Resources

About