Toward the Optimal Construction of a Loss Function Without Spurious Local Minima for Solving Quadratic Equations

Abstract

The problem of finding a vector x which obeys a set of quadratic equations |a ⊤ k x| 2 = y k , k = 1, · · · , m, plays an important role in many applications. In this paper we consider the case when both x and a k are real-valued vectors of length n. A new loss function is constructed for this problem, which combines the smooth quadratic loss function with an activation function. Under the Gaussian measurement model, we establish that with high probability the target solution x is the unique local minimizer (up to a global phase factor) of the new loss function provided m n. Moreover, the loss function always has a negative directional curvature around its saddle points.

Analysis of Asymptotic Escape of Strict Saddle Sets in Manifold Optimization

“…Surprisingly, we are able to prove that there are only a finite number of saddles and they are all strict saddles with very high probability. Our analysis provides a rigorous explanation for the robust performance of the PGD in solving the phase retrieval problem as a low-rank optimization problem, a phenomenon that has been observed in previous applications reported in the literature [41]. Although our primary focus is the low-rank matrix manifold, the asymptotic convergence to the minimum and the escape of strict saddles (strict critical submanifolds) are also valid on any arbitrary finite dimensional Riemannian manifold.…”

“…To ensure asymptotic convergence of the PGD to the global minimizer, it remains to rule out local minimizers and identify other critical points as strict saddles. Previous works [41], [57] have shown that phase retrieval has no spurious local minimum at least with high probability in the Euclidean setting. The analysis of saddles has been more complicated because of the stochasticity and Euclidean space parameterization.…”

“…, [41]. More importantly, they point out that the saddle points do not interfere with the convergence of various first-order algorithms towards a minimum, as long as they are``strict saddles,"" even though saddles are first-order stationary points [17], [30].…”

“…As an application of our asymptotic escape analysis for strict saddles, we consider the phase retrieval problem [15], [19], [29], [52] that has received considerable attention in recent years. We combine the perspectives of Riemannian manifold optimization [7] and landscape analysis [41], [57] and derive new results. We analyze the saddle points of the phase retrieval problem on the low-rank matrix manifold.…”

“…Asymptotic escape on the low-rank matrix manifold. In this section, we consider the phase retrieval problem [7], [41], [57] on the rank-1 matrix manifold. This serves both as an application of our asymptotic escape analysis for strict saddles and as a demonstration of the possibility of treating such problems rigorously on the manifold as opposed to the Euclidean space.…”