Analysis of Asymptotic Escape of Strict Saddle Sets in Manifold Optimization

Abstract

In this paper, we provide some analysis on the asymptotic escape of strict saddles in manifold optimization using the projected gradient descent algorithm (PGD). One of our main contributions is that we extend the current analysis to include nonisolated and possibly continuous saddle sets with complicated geometry. We prove that the PGD is able to escape strict critical submanifolds under certain conditions on the geometry and the distribution of the saddle point sets. We also show that the PGD may fail to escape strict saddles under weaker assumptions even if the saddle point set has zero measure. We apply this saddle analysis to the phase retrieval problem on the low-rank matrix manifold, prove that there are only a finite number of saddles, and that in a specific region, they are strict saddles with high probability. We also show the potential application of our analysis for a broader range of manifold optimization problems.

Solving Optimization Problems over the Stiefel Manifold by Smooth Exact Penalty Functions

“…With the frameworks and geometrical materials described in [4], theoretical results of these Riemannian optimization approaches have been established by following almost the same proof techniques as their unconstrained prototypes. These results include the global convergence, local convergence rate, worst-case complexity, and saddle-point-escaping properties, [5,12,13,24,32,55,69].…”