A New Modified Three-Term Hestenes–Stiefel Conjugate Gradient Method with Sufficient Descent Property and Its Global Convergence

Abstract

This paper describes a modified three-term Hestenes–Stiefel (HS) method. The original HS method is the earliest conjugate gradient method. Although the HS method achieves global convergence using an exact line search, this is not guaranteed in the case of an inexact line search. In addition, the HS method does not usually satisfy the descent property. Our modified three-term conjugate gradient method possesses a sufficient descent property regardless of the type of line search and guarantees global convergence using the inexact Wolfe–Powell line search. The numerical efficiency of the modified three-term HS method is checked using 75 standard test functions. It is known that three-term conjugate gradient methods are numerically more efficient than two-term conjugate gradient methods. Importantly, this paper quantifies how much better the three-term performance is compared with two-term methods. Thus, in the numerical results, we compare our new modification with an efficient two-term conjugate gradient method. We also compare our modification with a state-of-the-art three-term HS method. Finally, we conclude that our proposed modification is globally convergent and numerically efficient.

A New Three-Term Hestenes-Stiefel Type Method for Nonlinear Monotone Operator Equations and Image Restoration

“…This article modified and extended the work of Baluch et al [45] to solve nonlinear monotone operator equations. The modification become necessary so as to establish the descent and boundedness property of the search direction without the use of the line search.…”

“…The search direction generated by the algorithm is of three term and does not require the derivative of the operator. To have a good understanding of the motivation, the algorithm proposed by Baluch et al [45] for finding solution to the unconstrained optimization problem is recalled. Consider the unconstrained optimization problem:…”

“…where f : R n → R is a continuously differentiable real valued function with gradient at v (t) denoted by g (t) := ∇f (v (t) ). The algorithm proposed in [45] produces a sequence {v (t) } t≥0 via the following iterative formula…”

“…Inspired by the works in [43] and [44] as well as the good numerical performance of the Hestenes-Stiefel method, we modify and extend the three-term Hestenes-Stiefel method with sufficient descent property for unconstrained minimization problems proposed in [45]. The aim of the modification on the proposed search direction is to achieve the sufficient descent property and boundedness independent of the line search.…”

Optimal Homotopy Asymptotic Method-Least Square for Solving Nonlinear Fractional-Order Gradient-Based Dynamic System from an Optimization Problem

“…where f : R n ⟶ R is the objective function, h k ðxÞ: R n ⟶ R are equality constraint functions, g k ðxÞ: R n ⟶ R are inequality constraint functions, and C 2 are continuous differentiable functions. One of the main ideas of solving unconstrained NLP is by searching for the next point by choosing proper search direction d k and the stepsize α k as in the Newton direction [45], trust-region algorithm for unconstrained optimization [46]; the descent method [47], conjugate gradient method [48], three-term conjugate gradient method [49], and subspace method for nonlinear optimization [50]; the hybrid method for convex NLP [51]; CCM for optimization problem and application [52]; and descent direction stochastic approximation for optimization problem [53]. But there are studies for other approaches.…”