Hadamard Wirtinger Flow for Sparse Phase Retrieval

Abstract

We consider the problem of reconstructing an n-dimensional k-sparse signal from a set of magnitude-only measurements. Formulating the problem as an unregularized empirical risk minimization task, we study the sample complexity performance of gradient descent with Hadamard parametrization, which we call Hadamard Wirtinger flow (HWF). Provided knowledge of the signal sparsity k, we prove that a single step of HWF is able to recover the support from O(k(x * max ) −2 log n) samples, where x * max is the largest component of the signal in magnitude. This support recovery procedure can be used to initialize existing reconstruction methods and yields algorithms with total runtime proportional to the cost of reading the data and improved sample complexity, which is linear in k when the signal contains at least one large component. We numerically investigate the performance of HWF at convergence and show that, while not requiring any explicit form of regularization nor knowledge of k, HWF adapts to the signal sparsity and reconstructs sparse signals with fewer measurements than existing state-of-the-art methods.

Unrolled Wirtinger Flow with Deep Decoding Priors for Phaseless Imaging

“…If the step sizes of the gradient descent updates are fixed at µ ∈ R + for all l's and µ = µ y (0) 2 ≤ 2 β , then the inequality relation in (20) can be modified to…”

“…Several variants of WF have been introduced to improve its sample complexity of O(N log N ) [4], [19]. However, the exact recovery theory of original WF and all of its variants [4], [5], [18]- [20] relies on the assumption that the forward map is Gaussian. This poses a fundamental limitation for imaging applications since the forward models are almost always deterministic.…”

Robust Recovery via Implicit Bias of Discrepant Learning Rates for Double Over-parameterization

“…Learning over-parameterized models, which have more parameters than the problem's intrinsic dimension, is becoming a crucial topic in machine learning [1][2][3][4][5][6][7][8][9][10][11]. While classical learning theories suggest that over-parameterized models tend to overfit [12], recent advances showed that an optimization algorithm may produce an implicit bias that regularizes the solution with desired properties.…”

“…While classical learning theories suggest that over-parameterized models tend to overfit [12], recent advances showed that an optimization algorithm may produce an implicit bias that regularizes the solution with desired properties. This type of results has led to new insights and better understandings on gradient descent for solving several fundamental problems, including logistic regression on linearly separated data [1], compressive sensing [2,3], sparse phase retrieval [4], nonlinear least-squares [5], low-rank (deep) matrix factorization [6][7][8][9], and deep linear neural networks [10,11], etc.…”

“…Inspired by these recent advances [1][2][3][4][5][6][7][8][9][10][11], in this work we present a new type of practical methods for robust recovery of structured signals via model over-parameterization. In particular, we aim to learn an unknown signal X ∈ R n×n from its grossly corrupted linear measurements…”