A three-parameter family of nonlinear conjugate gradient methods

Abstract

Abstract. In this paper, we propose a three-parameter family of conjugate gradient methods for unconstrained optimization. The three-parameter family of methods not only includes the already existing six practical nonlinear conjugate gradient methods, but subsumes some other families of nonlinear conjugate gradient methods as its subfamilies. With Powell's restart criterion, the three-parameter family of methods with the strong Wolfe line search is shown to ensure the descent property of each search direction. Some general convergence results are also established for the three-parameter family of methods. This paper can also be regarded as a brief review on nonlinear conjugate gradient methods.

Modification of Nonlinear Conjugate Gradient Method with Weak Wolfe-Powell Line Search

“…The most popular formulas for are as follows: Hestenes-Stiefel (HS) [2], Fletcher-Reeves (FR) [3], PolakRibière-Polyak (PRP) [4], Conjugate Descent (CD) [5], LiuStorey (LS) [6], Dai-Yuan (DY) [7], Wei et al (WYL) [8], and Hager and Zhang (HZ) [9].…”

Global convergence of the Polak-Ribière-Polyak conjugate gradient method with an Armijo-type inexact line search for nonconvex unconstrained optimization problems

“…So for the CD method, (2.9) can be replaced by (2.11). It is known that if m = 0 and 1 − c = 0, then the CD formula (1.7) with the line search (2.8) and (2.11) need not converge; see [10] and [12] for details. Does the conjugate descent formula (1.7) with the line search conditions (2.8) and (2.11) converge globally?…”

“…The global convergence properties of the PRP method with (1.13) (P RP + ) and SWP has been widely discussed; see [3,5,9,11,12,15,17,25,30] for details. It is known from [9] and [12] that the P RP + with the SWP is globally convergent if g T k d k ≤ 0 for all k. In this remark, we will prove a similar result for the PRP with the MSWP, i.e., we prove the global convergence of the following algorithm if for all k, g…”

Memory gradient method with Goldstein line search

“…We can try to find some other parameters β (k) k− j for multi-step conjugate gradient methods [22,23]. However, the global convergence of this kind of methods cannot be guaranteed in many situations.…”