Accelerated Projected Gradient Method for Linear Inverse Problems with Sparsity Constraints

Abstract

Regularization of ill-posed linear inverse problems via 1 penalization has been proposed for cases where the solution is known to be (almost) sparse. One way to obtain the minimizer of such an 1 penalized functional is via an iterative softthresholding algorithm. We propose an alternative implementation to 1 -constraints, using a gradient method, with projection on 1 -balls. The corresponding algorithm uses again iterative soft-thresholding, now with a variable thresholding parameter. We also propose accelerated versions of this iterative method, using ingredients of the (linear) steepest descent method. We prove convergence in norm for one of these projected gradient methods, without and with acceleration.

Accelerated projected steepest descent method for nonlinear inverse problems with sparsity constraints

“…In what follows, we essentially proceed as in [3]. But as we shall see, several serious technical changes (including also a weakening of a few statements) but also significant extensions of the nice analysis provided in [3] need to be made.…”

“…The listed properties are proved here for completeness. They can be retraced in [3], from where they are partially taken, or to some extent in [4,5].…”

“…One first serious step to accelerate these kind of iterations was suggested in [3]. In [3] the authors "borrowed a leaf" from standard linear steepest descent methods by using an adaptive step length.…”

“…One first serious step to accelerate these kind of iterations was suggested in [3]. In [3] the authors "borrowed a leaf" from standard linear steepest descent methods by using an adaptive step length. In addition, a detailed analysis of the characteristic dynamics of the classical thresholded Landweber iteration (1.2) has shown that the algorithm converges initially relatively fast, then it overshoots the 1 penalty, and it takes very long to re-correct back.…”

“…A first intuitive way to circumvent this "external" detour is to force the iterates to remain within a particular 1 ball B R := {x ∈ 2 ; x 1 ≤ R}. The authors of [3] proposed to achieve this by substituting the thresholding operation S α by a projection P B R , where, for any closed convex set C and any x, the projection P C (x) is defined as the unique point in C for which the 2 distance to x is minimal. With denoting P B R by P R , this leads to the projected Landweber iteration, x n+1 = P R (x n + γF * (y − F x n )).…”