An asymptotically optimal gradient algorithm for quadratic optimization with low computational cost

Zhigljavsky, Anatoly; Pronzato, Luc; Bukina, Elena

doi:10.1007/s11590-012-0491-7

Cited by 9 publications

(11 citation statements)

References 10 publications

(16 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…According to (13), the larger i in M (i) k the more precise the estimation of M , which has a significant influence on the behavior of the algorithm, see [22]. Calculating high order moments has some computational cost, however, and a compromise must be made.…”

Section: Defining the Estimatorsmentioning

confidence: 99%

“…As shown in [22], using the largest β first in a pair (β 2j , β 2j+1 ) permits to improve the monotonicity of the algorithm (5). We shall thus also consider the case when…”

Section: Defining the Estimatorsmentioning

confidence: 99%

“…MINRES [11] (it coincides with that exhibited by CG and CR in the worst case, in terms of choice of the starting point and locations of the n − 2 internal eigenvalues of A in (m, M ), when the algorithm is stopped before n iterations). Such algorithms, with step-sizes generated externally, are simpler than CR and CG and thus of particular interest in situations where n is so large that the algorithm is stopped well before n iterations (in particular, it is shown in [22] that for some sequences of step-sizes the number of scalar products to be computed out of k iterations only grows as O(log k), see also Sect. 3.3 and 5).…”

Section: Introductionmentioning

confidence: 99%

“…The generation of suitable sequences of step-sizes γ k requires, however, the knowledge of the spectral bounds m and M . Since they are usually unknown, it is suggested in [22,16] to estimate them through the evaluation of moments of probability measures generated by the algorithm itself. It is the purpose of this paper to analyse the asymptotic properties of the estimators (1) of the leading eigenvalues of A.…”

Section: Introductionmentioning

confidence: 99%

“…random variables, (ii) they are constructed from a low discrepancy sequence. In addition, a more precise analysis of the behavior of the rate of convergence of the algorithm than in [22,16] will be provided (Sect. 4).…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Estimation of Spectral Bounds in Gradient Algorithms

2012

View full text Add to dashboard Cite

We consider the solution of linear systems of equations Ax = b, with A a symmetric positive-definite matrix in R n×n , through Richardson-type iterations or, equivalently, the minimization of convex quadratic functions (1/2)(Ax, x) − (b, x) with a gradient algorithm. The use of step-sizes asymptotically distributed with the arcsine distribution on the spectrum of A then yields an asymptotic rate of convergence after k < n iterations, k → ∞, that coincides with that of the conjugate-gradient algorithm in the worst case. However, the spectral bounds m and M are generally unknown and thus need to be estimated to allow the construction of simple and cost-effective gradient algorithms with fast convergence. It is the purpose of this paper to analyse the properties of estimators of m and M based on moments of probability measures ν k defined on the spectrum of A and generated by the algorithm on its way towards the optimal solution. A precise analysis of the behavior of the rate of convergence of the algorithm is also given. Two situations are considered: (i) the sequence of step-sizes corresponds to i.i.d. random variables, (ii) they are generated through a dynamical system (fractional parts of the golden ratio) producing a low-discrepancy sequence. In the first case, properties of random walk can be used to prove the convergence of simple spectral bound estimators based on the first moment of ν k . The second option requires a more careful choice of spectral bounds estimators but is shown to produce much less fluctuations for the rate of convergence of the algorithm.

show abstract

Section: Defining the Estimatorsmentioning

confidence: 99%

“…As shown in [22], using the largest β first in a pair (β 2j , β 2j+1 ) permits to improve the monotonicity of the algorithm (5). We shall thus also consider the case when…”

Section: Defining the Estimatorsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Estimation of Spectral Bounds in Gradient Algorithms

2012

View full text Add to dashboard Cite

show abstract

Robust integral compounding criteria for trend and correlation structures

Stehlík

López–Fidalgo

Casero-Alonso

et al. 2014

Stoch Environ Res Risk Assess

View full text Add to dashboard Cite

On R-linear convergence analysis for a class of gradient methods

Huang

2021

Comput Optim Appl

View full text Add to dashboard Cite

Gradient method is a simple optimization approach using minus gradient of the objective function as a search direction. Its efficiency highly relies on the choices of the stepsize. In this paper, the convergence behavior of a class of gradient methods, where the stepsize has an important property introduced in (Dai in Optimization 52: 2003), is analyzed. Our analysis is focused on minimization on strictly convex quadratic functions. We establish the R-linear convergence and derive an estimate for the R-factor. Specifically, if the stepsize can be expressed as a collection of Rayleigh quotient of the inverse Hessian matrix, we are able to show that these methods converge R-linearly and their R-factors are bounded above by 1 − 1 , where is the associated condition number. Preliminary numerical results demonstrate the tightness of our estimate of the R-factor.

show abstract

An asymptotically optimal gradient algorithm for quadratic optimization with low computational cost

Cited by 9 publications

References 10 publications

Estimation of Spectral Bounds in Gradient Algorithms

Estimation of Spectral Bounds in Gradient Algorithms

Robust integral compounding criteria for trend and correlation structures

On R-linear convergence analysis for a class of gradient methods

Contact Info

Product

Resources

About