A Family of Hybrid Stochastic Conjugate Gradient Algorithms for Local and Global Minimization Problems

Alnowibet, Khalid Abdulaziz; Mahdi, Salem; Alshamrani, Ahmad M.; Sallam, Karam M.; Mohamed, Ali Wagdy

doi:10.3390/math10193595

Cited by 3 publications

(6 citation statements)

References 93 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Conjugate gradient methods (CGs) are associated with a very strong global convergence theory for a local minimizer and they have low memory requirements. Moreover, in practice, combining the CG method with a line search strategy showed merit in dealing with an unconstrained minimization problem [20][21][22][23][24][25].…”

Section: Introductionmentioning

confidence: 99%

“…Recently, many novel formulas for determining the parameter β k have been suggested, with those corresponding to CG methods including two or three terms (see [21,22,25,51,52])…”

Section: Introductionmentioning

confidence: 99%

“…Many modifications and performance enhancements of the CG parameters have recently been proposed by numerous authors. Such recently developed GC parameters include two terms: the first one denotes the negative gradient vector, while the second term indicates the parameter proposed by the cited authors (see, for example, [21,22,25].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Family of Developed Hybrid Four-Term Conjugate Gradient Algorithms for Unconstrained Optimization with Applications in Image Restoration

Ali

Mahdi

2023

Symmetry

Self Cite

View full text Add to dashboard Cite

The most important advantage of conjugate gradient methods (CGs) is that these methods have low memory requirements and convergence speed. This paper contains two main parts that deal with two application problems, as follows. In the first part, three new parameters of the CG methods are designed and then combined by employing a convex combination. The search direction is a four-term hybrid form for modified classical CG methods with some newly proposed parameters. The result of this hybridization is the acquisition of a newly developed hybrid CGCG method containing four terms. The proposed CGCG has sufficient descent properties. The convergence analysis of the proposed method is considered under some reasonable conditions. A numerical investigation is carried out for an unconstrained optimization problem. The comparison between the newly suggested algorithm (CGCG) and five other classical CG algorithms shows that the new method is competitive with and in all statuses superior to the five methods in terms of efficiency reliability and effectiveness in solving large-scale, unconstrained optimization problems. The second main part of this paper discusses the image restoration problem. By using the adaptive median filter method, the noise in an image is detected, and then the corrupted pixels of the image are restored by using a new family of modified hybrid CG methods. This new family has four terms: the first is the negative gradient; the second one consists of either the HS-CG method or the HZ-CG method; and the third and fourth terms are taken from our proposed CGCG method. Additionally, a change in the size of the filter window plays a key role in improving the performance of this family of CG methods, according to the noise level. Four famous images (test problems) are used to examine the performance of the new family of modified hybrid CG methods. The outstanding clearness of the restored images indicates that the new family of modified hybrid CG methods has reliable efficiency and effectiveness in dealing with image restoration problems.

show abstract

Section: Introductionmentioning

confidence: 99%

“…Recently, many novel formulas for determining the parameter β k have been suggested, with those corresponding to CG methods including two or three terms (see [21,22,25,51,52])…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Family of Developed Hybrid Four-Term Conjugate Gradient Algorithms for Unconstrained Optimization with Applications in Image Restoration

Ali

Mahdi

2023

Symmetry

Self Cite

View full text Add to dashboard Cite

show abstract

“…Sundry's advanced methods provided fair results when numerically calculating the gradient vector values. See, for example, [34][35][36][37][43][44][45].…”

Section: Introductionmentioning

confidence: 99%

“…For example, the authors of [43,44] proposed a random mechanism for selecting the optimal finite-difference interval h and presented a good results when solving unconstrained minimization problems. However, when solving Problem (1), using the numerical differentiation methods to compute the approximate values of the Jacobian matrix is extremely expensive.…”

Section: Introductionmentioning

confidence: 99%

Adaptive Hybrid Mixed Two-Point Step Size Gradient Algorithm for Solving Non-Linear Systems

Ali

Mahdi

2023

Mathematics

Self Cite

View full text Add to dashboard Cite

In this paper, a two-point step-size gradient technique is proposed by which the approximate solutions of a non-linear system are found. The two-point step-size includes two types of parameters deterministic and random. A new adaptive backtracking line search is presented and combined with the two-point step-size gradient to make it globally convergent. The idea of the suggested method depends on imitating the forward difference method by using one point to estimate the values of the gradient vector per iteration where the number of the function evaluation is at most one for each iteration. The global convergence analysis of the proposed method is established under actual and limited conditions. The performance of the proposed method is examined by solving a set of non-linear systems containing high dimensions. The results of the proposed method is compared to the results of a derivative-free three-term conjugate gradient CG method that solves the same test problems. Fair, popular, and sensible evaluation criteria are used for comparisons. The numerical results show that the proposed method has merit and is competitive in all cases and superior in terms of efficiency, reliability, and effectiveness in finding the approximate solution of the non-linear systems.

show abstract

Biased stochastic conjugate gradient algorithm with adaptive step size for nonconvex problems

Huang,

Qin,

Liu

et al. 2024

Expert Systems with Applications

View full text Add to dashboard Cite

A Family of Hybrid Stochastic Conjugate Gradient Algorithms for Local and Global Minimization Problems

Cited by 3 publications

References 93 publications

A Family of Developed Hybrid Four-Term Conjugate Gradient Algorithms for Unconstrained Optimization with Applications in Image Restoration

A Family of Developed Hybrid Four-Term Conjugate Gradient Algorithms for Unconstrained Optimization with Applications in Image Restoration

Adaptive Hybrid Mixed Two-Point Step Size Gradient Algorithm for Solving Non-Linear Systems

Biased stochastic conjugate gradient algorithm with adaptive step size for nonconvex problems

Contact Info

Product

Resources

About