The Local Convexity of Solving Systems of Quadratic Equations

Abstract

This paper considers the recovery of a rank r positive semidefinite matrix XX T ∈ R n×n from m scalar measurements of the form yi := a T i XX T ai (i.e., quadratic measurements of X). Such problems arise in a variety of applications, including covariance sketching of highdimensional data streams, quadratic regression, quantum state tomography, among others. A natural approach to this problem is to minimize the loss function f (U ) = i (yi − a T i UU T ai) 2 which has an entire manifold of solutions given by {XO}O∈O r where Or is the orthogonal group of r × r orthogonal matrices; this is non-convex in the n × r matrix U , but methods like gradient descent are simple and easy to implement (as compared to semidefinite relaxation approaches). In this paper we show that once we have m ≥ Cnr log 2 (n) samples from isotropic gaussian ai, with high probability (a) this function admits a dimension-independent region of local strong convexity on lines perpendicular to the solution manifold, and (b) with an additional polynomial factor of r samples, a simple spectral initialization will land within the region of convexity with high probability. Together, this implies that gradient descent with initialization (but no re-sampling) will converge linearly to the correct X, up to an orthogonal transformation. We believe that this general technique (local convexity reachable by spectral initialization) should prove applicable to a broader class of nonconvex optimization problems.Mathematics Subject Classification. 65K99.

Solving Systems of Random Quadratic Equations via Truncated Amplitude Flow

“…Proposition 2 demonstrates that the distance of TAF's successive iterates to x is monotonically decreasing once the algorithm enters a small-size neighborhood around x. This neighborhood is commonly referred to as the basin of attraction; see further discussions in [19], [33], [6], [37], [39]. In other words, as soon as one lands within the basin of attraction, TAF's iterates remain in this region and will be attracted to x exponentially fast.…”

Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview

“…To better understand the challenge, recall the GD rule (59). When m is exceedingly large, the negative gradient concentrates around the population-level gradient, which forms a reliable search direction.…”

“…a constant fraction of measurements are outliers). It is obvious that the original GD iterates (59) are not robust, since the residual r t,i := (a i x t ) 2 − y i can be perturbed arbitrarily if i ∈ S. Hence, we instead include only a subset of the samples when forming the search direction, yielding a truncated GD update rule…”

Provable Low Rank Phase Retrieval

“…3) Tangentially related work: Some other tangentially related work includes: (i) computing the approximate rank r approximation of any matrix from its random sketches [37], [38]; (i) compressed covariance estimation using different sketching matrices for each data vector, but without the low-rank assumption [39]; and (iii) a generalization of low-rank covariance sketching [40]: it attempts to recover an n × r matrix U * from measurements y i = a i U * 2 with r n. When r = 1, this is the standard PR problem. In the general case, this is related to covariance sketching described above, but not to our problem.…”