In this paper, we address the subspace clustering problem. Given a set of data samples (vectors) approximately drawn from a union of multiple subspaces, our goal is to cluster the samples into their respective subspaces and remove possible outliers as well. To this end, we propose a novel objective function named Low-Rank Representation (LRR), which seeks the lowest rank representation among all the candidates that can represent the data samples as linear combinations of the bases in a given dictionary. It is shown that the convex program associated with LRR solves the subspace clustering problem in the following sense: When the data is clean, we prove that LRR exactly recovers the true subspace structures; when the data are contaminated by outliers, we prove that under certain conditions LRR can exactly recover the row space of the original data and detect the outlier as well; for data corrupted by arbitrary sparse errors, LRR can also approximately recover the row space with theoretical guarantees. Since the subspace membership is provably determined by the row space, these further imply that LRR can perform robust subspace clustering and error correction in an efficient and effective way.

Can we recover a complex signal from its Fourier magnitudes? More generally, given a set of m measurements, y k = |a * k x| for k = 1, . . . , m, is it possible to recover x ∈ C n (i.e., length-n complex vector)? This generalized phase retrieval (GPR) problem is a fundamental task in various disciplines, and has been the subject of much recent investigation. Natural nonconvex heuristics often work remarkably well for GPR in practice, but lack clear theoretical explanations. In this paper, we take a step towards bridging this gap. We prove that when the measurement vectors a k 's are generic (i.i.d. complex Gaussian) and numerous enough (m ≥ Cn log 3 n), with high probability, a natural least-squares formulation for GPR has the following benign geometric structure: (1) there are no spurious local minimizers, and all global minimizers are equal to the target signal x, up to a global phase; and (2) the objective function has a negative directional curvature around each saddle point. This structure allows a number of iterative optimization methods to efficiently find a global minimizer, without special initialization. To corroborate the claim, we describe and analyze a second-order trust-region algorithm.Provable methods for GPR. Although heuristic methods for GPR have been used effectively in practice [GS72, Fie82, SEC + 15, JEH15], only recently have researchers begun to develop methods with provable performance guarantees. The first results of this nature were obtained using semidefinite programming (SDP) relaxations [CESV13, CSV13, CL14, CLS15a, WdM15, VX14]. While this represented a substantial advance in theory, the computational complexity of semidefinite programming limits the practicality of this approach. 7 Recently, several provable nonconvex methods have been proposed for GPR.[NJS13] augmented the seminal error-reduction method [GS72] with spectral initialization and resampling to obtain the first provable nonconvex method for GPR. [CLS15b] studied the nonconvex formulation (1.1) under the same hypotheses as this paper, and showed that a combination of spectral initialization and local gradient descent recovers the true signal with near-optimal sample complexity. [CC15] worked with a different nonconvex formulation, and refined the spectral initialization and the local gradient descent with a step-adaptive truncation. With the modifications, they reduced the sample requirement to the optimal order. 8 More recent work in this line [ZCL16, ZL16, WGE16, KÖ16, GX16, BE16, Wal16] concerns error stability, alternative formulations, algorithms, and measurement models. Compared to the SDP-based methods, these methods are more scalable and closer to methods used in practice. All these analyses are based on local geometry in nature, and hence depend on the spectral initializer being sufficiently close to the target set. In contrast, we explicitly characterize the global function landscape of (1.1). Its benign global geometric structure allows several algorithmic choices (see Section 1.3) that need no special init...

No abstract

We consider the problem of recovering a complete (i.e., square and invertible) matrix A 0 , from Y ∈ R n×p with Y = A 0 X 0 , provided X 0 is sufficiently sparse. This recovery problem is central to theoretical understanding of dictionary learning, which seeks a sparse representation for a collection of input signals and finds numerous applications in modern signal processing and machine learning. We give the first efficient algorithm that provably recovers A 0 when X 0 has O (n) nonzeros per column, under suitable probability model for X 0 . In contrast, prior results based on efficient algorithms either only guarantee recovery when X 0 has O( √ n) zeros per column, or require multiple rounds of SDP relaxation to work when X 0 has O(n 1−δ ) nonzeros per column (for any constant δ ∈ (0, 1)).Our algorithmic pipeline centers around solving a certain nonconvex optimization problem with a spherical constraint. In this paper, we provide a geometric characterization of the objective landscape. In particular, we show that the problem is highly structured: with high probability, (1) there are no "spurious" local minimizers; and (2) around all saddle points the objective has a negative directional curvature. This distinctive structure makes the problem amenable to efficient optimization algorithms. In a companion paper [3], we design a second-order trust-region algorithm over the sphere that provably converges to a local minimizer from arbitrary initializations, despite the presence of saddle points.