Stable Principal Component Pursuit

Abstract

In this paper, we study the problem of recovering a low-rank matrix (the principal components) from a highdimensional data matrix despite both small entry-wise noise and gross sparse errors. Recently, it has been shown that a convex program, named Principal Component Pursuit (PCP), can recover the low-rank matrix when the data matrix is corrupted by gross sparse errors. We further prove that the solution to a related convex program (a relaxed PCP) gives an estimate of the low-rank matrix that is simultaneously stable to small entrywise noise and robust to gross sparse errors. More precisely, our result shows that the proposed convex program recovers the low-rank matrix even though a positive fraction of its entries are arbitrarily corrupted, with an error bound proportional to the noise level. We present simulation results to support our result and demonstrate that the new convex program accurately recovers the principal components (the low-rank matrix) under quite broad conditions. To our knowledge, this is the first result that shows the classical Principal Component Analysis (PCA), optimal for small i.i.d. noise, can be made robust to gross sparse errors; or the first that shows the newly proposed PCP can be made stable to small entry-wise perturbations.

On the linear convergence of the alternating direction method of multipliers

“…If the variable x is further constrained to be nonnegative, then the corresponding compressive sensing problem can be formulated as a three block (K = 3) convex separable optimization problem (1.1) by introducing a slack variable. Similarly, in the stable version of robust principal component analysis (PCA) [59], we are given an observation matrix M ∈ ℜ m×n which is a noise-corrupted sum of a low rank matrix L and a sparse matrix S. The goal is recover L and S by solving the following nonsmooth convex optimization problem minimize L * + ρ S 1 + λ Z 2 F subject to L + S + Z = M where · * denotes the matrix nuclear norm (defined as the sum of the matrix singular eigenvalues), while · 1 and · F denote, respectively, the ℓ 1 and the Frobenius norm of a matrix (equal to the standard ℓ 1 and ℓ 2 vector norms when the matrix is viewed as a vector). In the above formulation, Z denotes the noise matrix, and ρ, λ are some fixed penalty parameters.…”

Background Recovery by Fixed-Rank Robust Principal Component Analysis

“…1(a)). For Sets 2, 3, and 5 with non-sparse E, the empirical optimal λ (0.002) is smaller than the theoretical λ * (0.003), contrary to the theory of [13]. At this lower λ, the ranks of the optimal A recovered by LrALM are still larger than the known value of 1 ( Fig.…”

“…So the desired rank of the low-rank matrix is 1. LrALM and FrALM were tested on the test sets over a range of λ from 0.0001 to 0.5, including the theoretical optimal λ of √ m = 0.003, denoted as λ * , as proved in [13]. The parameters ρ and initial µ were set to the default values of 6 and 0.5/σ 1 , where σ 1 is the largest singular value of the initial Y, as for LrALM.…”

“…So, the value of λ directly influences the rank of A recovered by LrALM. Although Zhou et al [13] prove theoretically that the optimal λ can be set to 1/ max(m, n), this is true only if A is low-rank and E is sufficiently sparse. The parameter µ can also affect the accuracy of the recovered A by influencing the rank of A (see discussion below).…”

“…It constructs a data matrix from multiple views and decomposes it into a low-rank matrix that contains the background and a sparse matrix that captures nonbackground components. It has been proved that exact solution of RPCA problem is available if the data matrix is composed of a sufficiently low-rank matrix and a sufficiently sparse matrix [2,3,9,13]. Various algorithms have been proposed for solving RPCA problem [6,9,12].…”