ABSTRACT. Recent research indicates that many convex optimization problems with random constraints exhibit a phase transition as the number of constraints increases. For example, this phenomenon emerges in the 1 minimization method for identifying a sparse vector from random linear measurements. Indeed, the 1 approach succeeds with high probability when the number of measurements exceeds a threshold that depends on the sparsity level; otherwise, it fails with high probability.This paper provides the first rigorous analysis that explains why phase transitions are ubiquitous in random convex optimization problems. It also describes tools for making reliable predictions about the quantitative aspects of the transition, including the location and the width of the transition region. These techniques apply to regularized linear inverse problems with random measurements, to demixing problems under a random incoherence model, and also to cone programs with random affine constraints.The applied results depend on foundational research in conic geometry. This paper introduces a summary parameter, called the statistical dimension, that canonically extends the dimension of a linear subspace to the class of convex cones. The main technical result demonstrates that the sequence of intrinsic volumes of a convex cone concentrates sharply around the statistical dimension. This fact leads to accurate bounds on the probability that a randomly rotated cone shares a ray with a fixed cone. MOTIVATIONA phase transition is a sharp change in the character of a computational problem as its parameters vary. Recent research suggests that phase transitions emerge in many random convex optimization problems from mathematical signal processing and computational statistics; for example, see [DT09b, Sto09, OH10, CSPW11, DGM13, MT14b]. This paper proves that the locations of these phase transitions are determined by geometric invariants associated with the mathematical programs. Our analysis provides the first complete account of transition phenomena in random linear inverse problems, random demixing problems, and random cone programs.1.1. Vignette: Compressed sensing. To illustrate our goals, we discuss the compressed sensing problem, a familiar example where a phase transition is plainly visible in numerical experiments [DT09b]. Let x 0 ∈ R d be an unknown vector with s nonzero entries. Let A be an m × d random matrix whose entries are independent standard normal variables, and suppose we have access to the vector(1.1)This serves as a model for data acquisition: we interpret z 0 as a collection of m independent linear measurements of the unknown x 0 . The compressed sensing problem requires us to identify x 0 given only the measurement vector z 0 and the realization of the measurement matrix A. When the number m of measurements is smaller than the ambient dimension d , we cannot solve this inverse problem unless we take advantage of the prior knowledge that x 0 is sparse. |x i |. This approach is sensible because the 1 norm of a vector can serv...

No abstract

Consider a dataset of vector-valued observations that consists of noisy inliers, which are explained well by a low-dimensional subspace, along with some number of outliers. This work describes a convex optimization problem, called REAPER, that can reliably fit a low-dimensional model to this type of data. This approach parameterizes linear subspaces using orthogonal projectors, and it uses a relaxation of the set of orthogonal projectors to reach the convex formulation. The paper provides an efficient algorithm for solving the REAPER problem, and it documents numerical experiments which confirm that REAPER can dependably find linear structure in synthetic and natural data. In addition, when the inliers lie near a low-dimensional subspace, there is a rigorous theory that describes when REAPER can approximate this subspace.

The performance of principal component analysis (PCA) suffers badly in the presence of outliers. This paper proposes two novel approaches for robust PCA based on semidefinite programming. The first method, maximum mean absolute deviation rounding (MDR), seeks directions of large spread in the data while damping the effect of outliers. The second method produces a low-leverage decomposition (LLD) of the data that attempts to form a low-rank model for the data by separating out corrupted observations. This paper also presents efficient computational methods for solving these SDPs. Numerical experiments confirm the value of these new techniques.

Demixing refers to the challenge of identifying two structured signals given only the sum of the two signals and prior information about their structures. Examples include the problem of separating a signal that is sparse with respect to one basis from a signal that is sparse with respect to a second basis, and the problem of decomposing an observed matrix into a low-rank matrix plus a sparse matrix. This paper describes and analyzes a framework, based on convex optimization, for solving these demixing problems, and many others. This work introduces a randomized signal model which ensures that the two structures are incoherent, i.e., generically oriented. For an observation from this model, this approach identifies a summary statistic that reflects the complexity of a particular signal. The difficulty of separating two structured, incoherent signals depends only on the total complexity of the two structures. Some applications include (i) demixing two signals that are sparse in mutually incoherent bases; (ii) decoding spread-spectrum transmissions in the presence of impulsive errors; and (iii) removing sparse corruptions from a low-rank matrix. In each case, the theoretical analysis of the convex demixing method closely matches its empirical behavior.Communicated by Emmanuel Candès.