A cyclic projected gradient method

Abstract

In recent years, convex optimization methods were successfully applied for various image processing tasks and a large number of first-order methods were designed to minimize the corresponding functionals. Interestingly, it was shown recently in [24] that the simple idea of so-called "superstep cycles" leads to very efficient schemes for time-dependent (parabolic) image enhancement problems as well as for steady state (elliptic) image compression tasks. The "superstep cycles" approach is similar to the nonstationary (cyclic) Richardson method which has been around for over sixty years.In this paper, we investigate the incorporation of superstep cycles into the projected gradient method. We show for two problems in compressive sensing and image processing, namely the LASSO approach and the Rudin-Osher-Fatemi model that the resulting simple cyclic projected gradient algorithm can numerically compare with various state-of-the-art first-order algorithms. However, due to the nonlinear projection within the algorithm convergence proofs even under restrictive assumptions on the linear operators appear to be hard. We demonstrate the difficulties by studying the simplest case of a two-cycle algorithm in R 2 with projections onto the Euclidean ball.

First Order Algorithms in Variational Image Processing

“…Line search strategies can be incorporated [83,87,120]. Finally we mention Barzilei-Borwein step size rules [11] based on a Quasi-Newton approach and relatives, see [74] for an overview and the cyclic proximal gradient algorithm related to the cyclic Richardson algorithm [158].…”

An Optimally Generalized Steepest-Descent Algorithm for Solving Ill-Posed Linear Systems

“…where Δr −1 = r − r −1 and Δx −1 = x − x −1 , with initial guesses r 0 = 0 and x 0 = 0. Nowadays the improvements of the original BBM have been developed in many literature works [24][25][26][27][28][29][30] to treat different ill-posed and inverse problems.…”

“…Upon comparing (53) and (30) we can derive (37) with the steplength being positive. Moreover, as a direct result of (30) and (37) we have r ⋅ u > 0.…”

Limited-memory scaled gradient projection methods for real-time image deconvolution in microscopy

“…In [3] numerical evidence has been also provided indicating remarkable gain in the convergence rate over the classical BarzilaiBorwein (BB) step-length rule [4]. Since in the last years promising image reconstruction algorithms have been designed by exploiting BB-based rules within gradient methods [5,6,7,8,9,10,11], it is worthwhile to investigate if useful acceleration can be achieved with the new step-length selection idea. In particular, we focus on the algorithm for image deconvolution in microscopy provided by the Scaled Gradient Projection (SGP) method recently developed in [12], that can be appropriately modified for managing the step-length rule proposed in [3].…”