An efficient modification of the Hestenes-Stiefel nonlinear conjugate gradient method with restart property

Abstract

The conjugate gradient (CG) method is one of the most popular methods to solve nonlinear unconstrained optimization problems. The Hestenes-Stiefel (HS) CG formula is considered one of the most efficient methods developed in this century. In addition, the HS coefficient is related to the conjugacy condition regardless of the line search method used. However, the HS parameter may not satisfy the global convergence properties of the CG method with the Wolfe-Powell line search if the descent condition is not satisfied. In this paper, we use the original HS CG formula with a mild condition to construct a CG method with restart using the negative gradient. The convergence and descent properties with the strong Wolfe-Powell (SWP) and weak Wolfe-Powell (WWP) line searches are established. Using this condition, we guarantee that the HS formula is non-negative, its value is restricted, and the number of restarts is not too high. Numerical computations with the SWP line search and some standard optimization problems demonstrate the robustness and efficiency of the new version of the CG parameter in comparison with the latest and classical CG formulas. An example is used to describe the benefit of using different initial points to obtain different solutions for multimodal optimization functions.

Preconditioned nonlinear conjugate gradient method for micromagnetic energy minimization

“…The preconditioned nonlinear conjugate gradient method for micromagnetic energy minimization is summarized in Algorithm 4. (∇F (x j+1 )) Solve P j+1 y j+1 = g j+1 Compute search direction d j+1 = −y j+1 + β * ZA d j end for Algorithm 4 is the Hestenes-Stiefel conjugate gradient method with restarts [39] according to…”

Global Convergence Property with Inexact Line Search for a New Hybrid Conjugate Gradient Method

“…The new scale parameter φ satisfies the sufficient descent condition, global convergence analysis proved under Strong Wolfe line search conditions. Our numerical results show that the proposed method is effective and robust against some known algorithms.search, known as the line searches [2]. Among them, the so-called strong wolf line search conditions require that [3] [4].…”

“…search, known as the line searches [2]. Among them, the so-called strong wolf line search conditions require that [3] [4].…”

Modification of Nonlinear Conjugate Gradient Method with Weak Wolfe-Powell Line Search

“…Therefore, PRP method is the most efficient method when it is compared to the other conjugate gradient methods. For more, the reader can see the following references [14][15][16][17][18][19].…”