Iteratively reweighted least squares minimization for sparse recovery

Abstract

Under certain conditions (known as the Restricted Isometry Property or RIP) on the m × Nmatrix Φ (where m < N ), vectors x ∈ R N that are sparse (i.e. have most of their entries equal to zero) can be recovered exactly from y := Φx even though Φ −1 (y) is typically an (N − m)-dimensional hyperplane; in addition x is then equal to the element in Φ −1 (y) of minimal ℓ 1 -norm. This minimal element can be identified via linear programming algorithms. We study an alternative method of determining x, as the limit of an Iteratively Re-weighted Least Squares (IRLS) algorithm. The main step of this IRLS finds, for a given weight vector w, the element in Φ −1 (y) with smallest ℓ 2 (w)-norm. If x (n) is the solution at iteration step n,, i = 1, . . . , N , for a decreasing sequence of adaptively defined ǫ n ; this updated weight is then used to obtain x (n+1) and the process is repeated. We prove that when Φ satisfies the RIP conditions, the sequence xconverges for all y, regardless of whether Φ −1 (y) contains a sparse vector. If there is a sparse vector in Φ −1 (y), then the limit is this sparse vector, and when x (n) is sufficiently close to the limit, the remaining steps of the algorithm converge exponentially fast (linear convergence in the terminology of numerical optimization). The same algorithm with the "heavier" weight, i = 1, . . . , N , where 0 < τ < 1, can recover sparse solutions as well; more importantly, we show its local convergence is superlinear and approaches a quadratic rate for τ approaching to zero.

Joint channel estimation algorithm via weighted Homotopy for massive MIMO OFDM system

“…To improve the recovery performance, it naturally leads us to substitute the ℓ 1 norm or ℓ 2 norm in (5) for a weighted norm (see more details for the weighted ℓ 2 norm case in [16]- [18]). According to the system model discussed in Section 2, the weighted ℓ 1 norm minimization problem could be described as…”

“…Existing literature 60 includes several improvements proposed for the standard Homotopy algorithm [14,15,16,17,18]. One of the latest researches is the weighted Homotopy in [15], which improves the original Homotopy by replacing its ℓ 1 term with a weighted ℓ 1 term.…”

Mixed <i>l</i>₀/<i>l</i>₁ Norm Minimization Approach to Image Colorization

“…Since the problem (7) is non-convex and difficult to solve exactly, this letter applies the iteratively reweighted least squares (IRLS) [8], [9]. The IRLS provides an approximate solution of the l p norm minimization of z = [z 1 z 2 .…”

“…Candès et al proposes the reweighted l 1 minimization to obtain a sparse vector and apply to the image recover problem [8]. Motivated by their work we formulate an image colorization problem as the mixed l 0 /l 1 norm minimization instead of the TV norm minimization, and apply the iteratively reweighted least squares (IRLS) [9] to recover a color image. The proposed algorithm can recover the color image with small given color regions or the small number of given color pixels.…”

A sampling theory for infinite weighted graphs

“…There is a recent renewed interest in sampling techniques, motivated in part by quantization requirements in digital signal processing, and by contrast to our present approach, much of this is based on harmonic analysis tools, see e.g., [1,2,6,13,14], and other investigations on Markov processes [15, 18-20, 22, 23, 26].…”