The seminal paper by Barzilai and Borwein (1988) has given rise to an extensive investigation, leading to the development of effective gradient methods. Several steplength rules have been first designed for unconstrained quadratic problems and then extended to general nonlinear optimization problems. These rules share the common idea of attempting to capture, in an inexpensive way, some second-order information. However, the convergence theory of the gradient methods using the previous rules does not explain their effectiveness, and a full understanding of their practical behaviour is still missing. In this work we investigate the relationships between the steplengths of a variety of gradient methods and the spectrum of the Hessian of the objective function, providing insight into the computational effectiveness of the methods, for both quadratic and general unconstrained optimization problems. Our study also identifies basic principles for designing effective gradient methods

Variational models are a valid tool for edge-preserving image restoration from data affected by Poisson noise. This paper deals with total variation and hypersurface regularization in combination with the Kullbach Leibler divergence as data fidelity function. We propose an iterative method, based on an alternating extragradient scheme, which is able to solve in a numerically stable way the primaldual formulation of both total variation and hypersurface regularization problems. In this method, tailored for general smooth saddle-point problems, the stepsize parameter can be adaptively computed so that the convergence of the scheme is proved under mild assumptions. In the numerical experience, we focus the attention on the artificial smoothing parameter that makes different the total variation and hypersurface regularization. A set of experiments on image denoising and deblurring problems is performed in order to evaluate the influence of this smoothing parameter on the stability of the proposed method and on the features of the restored images.

No abstract

Forward-backward methods are a very useful tool for the minimization of a functional given by the sum of a differentiable term and a nondifferentiable one, and their investigation hascomprised several efforts from many researchers in the last decade. In this paper we focus on the convex case and, inspired by recent approaches for accelerating first-order iterative schemes, we\ud develop a scaled inertial forward-backward algorithm which is based on a metric changing at each iteration and on a suitable extrapolation step. Unlike standard forward-backward methods with extrapolation, our scheme is able to handle functions whose domain is not the entire space. Both an O(1/k^2) convergence rate estimate on the objective function values and the convergence of the sequence of the iterates are proved. Numerical experiments on several test problems arising from\ud image processing, compressed sensing, and statistical inference show the effectiveness of the proposed method in comparison to well-performing state-of-the-art algorithms

This paper is concerned with the numerical solution of a symmetric indefinite system which is a generalization of the Karush Kuhn Tucker system. Following the recent approach of Luk.san and Vl.cek, we propose to solve this system by a preconditioned conjugate gradient (PCG) algorithm and we devise two indefin ite preconditioners with good theoretical properties. In particular, for one of these preconditioners, the finite termination property of the PCG method is stated. The PCG method combined\ud with a parallel version of these preconditioners is used as inner solver within an inexact Interior-Point (IP) method for the solution of large and sparse quadratic programs. The numerical results obtained by a parallel code implementing the IP method on distributed memory multiprocessor systems enable us to confirm the effectiveness of the proposed approach for problems with special structure in the constraint matrix and in the objective function