On R-linear convergence analysis for a class of gradient methods

Abstract

Gradient method is a simple optimization approach using minus gradient of the objective function as a search direction. Its efficiency highly relies on the choices of the stepsize. In this paper, the convergence behavior of a class of gradient methods, where the stepsize has an important property introduced in (Dai in Optimization 52: 2003), is analyzed. Our analysis is focused on minimization on strictly convex quadratic functions. We establish the R-linear convergence and derive an estimate for the R-factor. Specifically, if the stepsize can be expressed as a collection of Rayleigh quotient of the inverse Hessian matrix, we are able to show that these methods converge R-linearly and their R-factors are bounded above by 1 − 1 , where is the associated condition number. Preliminary numerical results demonstrate the tightness of our estimate of the R-factor.

A gradient method exploiting the two dimensional quadratic termination property

“…In [24], the authors prove that any gradient method with stepsizes satisfying the following Property B has R-linear convergence rate 1 − λ 1 /M 1 which implies a 1 − 1/κ rate when M 1 ≤ λ n . Similar results for gradient methods satisfying the Property A in [7] can be found in [23]. However, a stepsize satisfies Property B may not meets the conditions of Property A.…”