A Family of Multi-Step Subgradient Minimization Methods

Tovbis, Elena; Крутиков, В. Н.; Stanimirović, Predrag S.; Мешечкин, Владимир Викторович; Popov, A. A.; Kazakovtsev, Lev

doi:10.3390/math11102264

Cited by 1 publication

(3 citation statements)

References 67 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We implemented and compared quasi-Newtonian BFGS and DFP methods. A onedimensional search procedure with cubic interpolation [41] (exact one-dimensional descent) and a one-dimensional minimization procedure [34] (inexact one-dimensional descent) were used. We used both the classical QNM with the iterations of (1)-(4) (denoted as BFGS and DFP) and the QNM including iterations with additional orthogonalization (116)-(119) in the form of a sequence of iterations (120) (denoted as BFGS_V and DFP_V).…”

Section: Numerical Study Of Ways To Increase the Orthogonality Of Lea...mentioning

confidence: 99%

“…In this paper, we consider a method for deriving matrix updating equations in QNMs by forming a quality functional based on learning relations for matrices, followed by obtaining matrix updating equations in the form of a step of the gradient method for minimizing the quality functional. This approach has shown high efficiency in organizing subgradient minimization methods [34,35].…”

Section: Introductionmentioning

confidence: 99%

“…., n − 1, in (13) be given, vectors y k /∥y k ∥ be columns of matrix P, and the minimum eigenvalue µ of a matrix P T P satisfy the constraint µ ≥ µ 0 > 0. Then, to estimate the convergence rate of the process in (34), the following inequality holds:…”

mentioning

confidence: 99%

See 2 more Smart Citations

Machine Learning in Quasi-Newton Methods

Krutikov,

Tovbis,

Stanimirović

et al. 2024

Axioms

Self Cite

View full text Add to dashboard Cite

In this article, we consider the correction of metric matrices in quasi-Newton methods (QNM) from the perspective of machine learning theory. Based on training information for estimating the matrix of the second derivatives of a function, we formulate a quality functional and minimize it by using gradient machine learning algorithms. We demonstrate that this approach leads us to the well-known ways of updating metric matrices used in QNM. The learning algorithm for finding metric matrices performs minimization along a system of directions, the orthogonality of which determines the convergence rate of the learning process. The degree of learning vectors’ orthogonality can be increased both by choosing a QNM and by using additional orthogonalization methods. It has been shown theoretically that the orthogonality degree of learning vectors in the Broyden–Fletcher–Goldfarb–Shanno (BFGS) method is higher than in the Davidon–Fletcher–Powell (DFP) method, which determines the advantage of the BFGS method. In our paper, we discuss some orthogonalization techniques. One of them is to include iterations with orthogonalization or an exact one-dimensional descent. As a result, it is theoretically possible to detect the cumulative effect of reducing the optimization space on quadratic functions. Another way to increase the orthogonality degree of learning vectors at the initial stages of the QNM is a special choice of initial metric matrices. Our computational experiments on problems with a high degree of conditionality have confirmed the stated theoretical assumptions.

show abstract

Section: Numerical Study Of Ways To Increase the Orthogonality Of Lea...mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%