On the Estimation Performance and Convergence Rate of the Generalized Power Method for Phase Synchronization

Liu, Huikang; Yue, Man–Chung; So, Anthony Man–Cho

doi:10.1137/16m110109x

Cited by 77 publications

(100 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This coincides with the setting studied in [71,82] over the orthogonal group SO.2/, under the name of synchronization [3,9,55]. It has been shown that the leading eigenvector of a certain data matrix becomes positively correlated with the truth as long as 0 > 1= p n when p obs D 1 [71].…”

Section: Extension: Large-m Casesupporting

confidence: 79%

The Projected Power Method: An Efficient Algorithm for Joint Alignment from Pairwise Differences

Chen

Candès

2018

Comm Pure Appl Math

115

View full text Add to dashboard Cite

Various applications involve assigning discrete label values to a collection of objects based on some pairwise noisy data. Due to the discrete—and hence nonconvex—structure of the problem, computing the optimal assignment (e.g., maximum‐likelihood assignment) becomes intractable at first sight. This paper makes progress towards efficient computation by focusing on a concrete joint alignment problem; that is, the problem of recovering n discrete variables xi ∊ {1, …, m}, 1 ≤ i ≤ n, given noisy observations of their modulo differences {xi — xj mod m}. We propose a low‐complexity and model‐free nonconvex procedure, which operates in a lifted space by representing distinct label values in orthogonal directions and attempts to optimize quadratic functions over hypercubes. Starting with a first guess computed via a spectral method, the algorithm successively refines the iterates via projected power iterations. We prove that for a broad class of statistical models, the proposed projected power method makes no error—and hence converges to the maximum‐likelihood estimate—in a suitable regime. Numerical experiments have been carried out on both synthetic and real data to demonstrate the practicality of our algorithm. We expect this algorithmic framework to be effective for a broad range of discrete assignment problems.© 2018 Wiley Periodicals, Inc.

show abstract

Section: Extension: Large-m Casesupporting

confidence: 79%

The Projected Power Method: An Efficient Algorithm for Joint Alignment from Pairwise Differences

Chen

Candès

2018

Comm Pure Appl Math

115

View full text Add to dashboard Cite

show abstract

“…• Phase synchronization [92,93]. Suppose we wish to recover n unknown phases φ 1 , · · · , φ n ∈ [0, 2π] given their pairwise relative phases.…”

Section: Projected Power Methods For Constrained Pcamentioning

confidence: 99%

Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview

Chi

Chen

2019

IEEE Trans. Signal Process.

324

284

View full text Add to dashboard Cite

Substantial progress has been made recently on developing provably accurate and efficient algorithms for low-rank matrix factorization via nonconvex optimization. While conventional wisdom often takes a dim view of nonconvex optimization algorithms due to their susceptibility to spurious local minima, simple iterative methods such as gradient descent have been remarkably successful in practice. The theoretical footings, however, had been largely lacking until recently.In this tutorial-style overview, we highlight the important role of statistical models in enabling efficient nonconvex optimization with performance guarantees. We review two contrasting approaches: (1) two-stage algorithms, which consist of a tailored initialization step followed by successive refinement; and (2) global landscape analysis and initialization-free algorithms. Several canonical matrix factorization problems are discussed, including but not limited to matrix sensing, phase retrieval, matrix completion, blind deconvolution, robust principal component analysis, phase synchronization, and joint alignment. Special care is taken to illustrate the key technical insights underlying their analyses. This article serves as a testament that the integrated consideration of optimization and statistics leads to fruitful research findings.

show abstract

“…It follows from (29) that φ is either a zero vector (i.e., φ = 0), or a left singular vector of Y (i.e., φ = αp j for some j ∈ [r]). Plugging φ = αp j into (29) gives…”

Section: A Proof Of Lemmamentioning

confidence: 99%

“…Outside of the context of neural networks, such geometric analysis (characterizing the behavior of all critical points) has been recognized as a powerful tool for understanding nonconvex optimization problems in applications such as phase retrieval [35,40], dictionary learning [41], tensor factorization [12], phase synchronization [29] and low-rank matrix optimization [3,13,14,25,26,34,45,46]. A similar regularizer (see (6)) to the one used in (2) is also utilized in [13,25,34,45,46] for analyzing the optimization geometry.…”

Section: Introductionmentioning

confidence: 99%

The Global Optimization Geometry of Shallow Linear Neural Networks

et al. 2019

View full text Add to dashboard Cite

We examine the squared error loss landscape of shallow linear neural networks. We show-with significantly milder assumptions than previous worksthat the corresponding optimization problems have benign geometric properties: there are no spurious local minima and the Hessian at every saddle point has at least one negative eigenvalue. This means that at every saddle point there is a directional negative curvature which algorithms can utilize to further decrease the objective value. These geometric properties imply that many local search algorithms (such as the gradient descent which is widely utilized for training neural networks) can provably solve the training problem with global convergence. 1From an optimization perspective, non-strict saddle points and local minima have similar first-/second-order information and it is hard for first-/second-order methods (like gradient descent) to distinguish between them.

show abstract

On the Estimation Performance and Convergence Rate of the Generalized Power Method for Phase Synchronization

Cited by 77 publications

References 23 publications

The Projected Power Method: An Efficient Algorithm for Joint Alignment from Pairwise Differences

The Projected Power Method: An Efficient Algorithm for Joint Alignment from Pairwise Differences

Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview

The Global Optimization Geometry of Shallow Linear Neural Networks

Contact Info

Product

Resources

About