Smoothed analysis of the low-rank approach for smooth semidefinite programs

Abstract

We consider semidefinite programs (SDPs) of size n with equality constraints. In order to overcome scalability issues, Burer and Monteiro proposed a factorized approach based on optimizing over a matrix Y of size n × k such that X = Y Y * is the SDP variable. The advantages of such formulation are twofold: the dimension of the optimization variable is reduced, and positive semidefiniteness is naturally enforced. However, optimization in Y is non-convex. In prior work, it has been shown that, when the constraints on the factorized variable regularly define a smooth manifold, provided k is large enough, for almost all cost matrices, all second-order stationary points (SOSPs) are optimal. Importantly, in practice, one can only compute points which approximately satisfy necessary optimality conditions, leading to the question: are such points also approximately optimal? To answer it, under similar assumptions, we use smoothed analysis to show that approximate SOSPs for a randomly perturbed objective function are approximate global optima, with k scaling like the square root of the number of constraints (up to log factors). Moreover, we bound the optimality gap at the approximate solution of the perturbed problem with respect to the original problem. We particularize our results to an SDP relaxation of phase retrieval.

Solving graph equipartition SDPs on an algebraic variety

“…We also prove that under the constraint nondegeneracy condition [29], any local minimum of the factorized problem is a global optimal solution with probability 1. These results generalize the results of Boumal et al from [3,5,6,24] since we have an extra non-smooth inequality constraint and our feasible region is not necessarily a smooth manifold. Our study here may serve as a prototype for extending the nonconvex factorization approach for solving other types of SDP problems whose resulting feasible regions may not be smooth manifolds.…”

“…We will also prove that under the constraint nondegeneracy condition see [29], for almost all C ∈ S n except for a set of measure zero, any local minimum of ( 6) is also a global optimal solution. This is a generalization of the results in [5,6,24]. If we choose A(•) = diag(•) and C = L, then ( 6) is the low-rank factorization of (4) for α = n k−1 , and we get…”

“…JCb. These equations implies that ( 26) satisfies (24). Now, suppose R ∈ B 1 n,r , we have Proj R(C ) := arg min…”

“…Since ( 2) is a non-convex problem, it may have spurious local minima. As far as we know, the first foundational and thorough analysis about the global optimality of ( 2) is in a series of works [3,5,6,24], by Boumal et al They proved that under LICQ-like regularity assumptions on the constraints of (2), if r ≥ √ 2m, then for almost all C ∈ S n except for a set of measure zero, every local minima of ( 2) is a global optimal solution. This result explains why the BM's approach can almost always find the global minimum in practice.…”

“…The research by Boumal et al in [3,5,6,24] was a milestone in studying the global optimality of the non-convex factorized formulation of a linear SDP problem by combining the geometric and low-rank property of the SDP problem. Their results have since been generalized by Cifuentes in [10] to linear SDP problems with multiple positive semidefinite block variables and inequality constraints.…”