Nearly minimax-optimal rates for noisy sparse phase retrieval via early-stopped mirror descent

Abstract

This paper studies early-stopped mirror descent applied to noisy sparse phase retrieval, which is the problem of recovering a $k$-sparse signal $\textbf{x}^\star \in{\mathbb{R}}^n$ from a set of quadratic Gaussian measurements corrupted by sub-exponential noise. We consider the (non-convex) unregularized empirical risk minimization problem and show that early-stopped mirror descent, when equipped with the hypentropy mirror map and proper initialization, achieves a nearly minimax-optimal rate of convergence, provided the sample size is at least of order $k^2$ (modulo logarithmic term) and the minimum (in modulus) non-zero entry of the signal is on the order of $\|\textbf{x}^\star \|_2/\sqrt{k}$. Our theory leads to a simple algorithm that does not rely on explicit regularization or thresholding steps to promote sparsity. More generally, our results establish a connection between mirror descent and sparsity in the non-convex problem of noisy sparse phase retrieval, adding to the literature on early stopping that has mostly focused on non-sparse, Euclidean and convex settings via gradient descent. Our proof combines a potential-based analysis of mirror descent with a quantitative control on a variational coherence property that we establish along the path of mirror descent, up to a prescribed stopping time.

Provable sample-efficient sparse phase retrieval initialized by truncated power method

“…Especially, when there are only very few significant components in x such that sx = O(1), the sample complexity of algorithm 1 is m = Ω(s log n), which is optimal in s and order-wisely better than m = Ω(s 2 log(n)) in [25,50]. Recently, [53] proposed a new initialization method that is a multiple run of algorithm 2 combined with a few steps of Hardmard Wirtinger flow, and the sample complexity of its initialization is m = Ω(δ −2 sx s log(n)) under the assumption that min i∈S |x i | ⩾ ∥x∥ ℓ2 / √ s. On the contrary, algorithm 1 not only achieves the same order of sample complexity without any assumption on x, but also improves dependency on δ and sx from δ −2 sx to max{δ −2 ,s x }. Since δ is usually a small constant in various non-convex algorithms, this improvement is significant, which makes algorithm 1 initialized algorithms outperform others.…”

“…Consequently, it is possible to break the sample complexity barriers as long as the methods for initialization can be improved. Alternatively, a Hadamard Wirtinger flow (HWF) method has been introduced in [53] as a different strategy to obtain the initial guess, where an implicit regularization is used. The sample complexity of HWF is improved to Ω(s x s log n) under the assumption that x min = Ω( 1 √ s ∥x∥ ℓ2 ), where ∥ • ∥ ℓ2 denotes the ℓ 2 -norm for vectors, sx :=…”

“…Recall that the spectral initialization first estimates the support of x by taking the indices set S of the top s diagonal entries of Y (see ( 9) and ( 10)) and then use a rescaled eigenvector of [ Ȳ] S to approximate non-zero entries of x. Following [53], instead of the diagonal entries, we use the indices of the s largest number of entries of Ye j0 in absolute value as S, where j 0 is the index that Y jj achieves maximum. Our choice of S is based on the observation that…”

“…We obtain a modified spectral method summarized in algorithm 2. A similar idea by using the set with top s elements of Ye j0 in absolute value was adopted in [53]. However, their theory was established based on some lower bound assumption on x min , which allows only a very small dynamic range of the nonzero entries of the underlying signal.…”

“…When combined with the local refinement algorithm HTP [11, algorithm 1], we then have the two-stage algorithm (described in algorithm 3 for completeness). Also, to empirically get a better estimate, similar to [53], for the truncated power method (TP) and modified spectral method, we can further implement a multiple-restarted version. The details are given in algorithm 4.…”