Optimal-order convergence of Nesterov acceleration for linear ill-posed problems*

Kindermann, Stefan

doi:10.1088/1361-6420/abf5bc

Cited by 11 publications

(5 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Inspired by this, we investigate in this paper how AA(m) for linear problems, with β (k) given by (1.8), relates to Krylov methods. Following [11,13,16], this is easy to see for AA(1) applied to (1.11), as we now explain. Given x 0 , let x 1 = q(x 0 ).…”

Section: Introductionmentioning

confidence: 77%

See 1 more Smart Citation

Anderson Acceleration as a Krylov Method with Application to Asymptotic Convergence Analysis

Sterck

2021

Preprint

View full text Add to dashboard Cite

Anderson acceleration is widely used for accelerating the convergence of fixed-point methods x k+1 = q(x k ), x k ∈ R n . We consider the case of linear fixed-point methods x k+1 = M x k +b and obtain polynomial residual update formulas for AA(m), i.e., Anderson acceleration with window size m. We find that the standard AA(m) method with initial iterates x k , k = 0, . . . , m defined recursively using AA(k), is a Krylov space method. This immediately implies that k iterations of AA(m) cannot produce a smaller residual than k iterations of GMRES without restart (but without implying anything about the relative convergence speed of (windowed) AA(m) versus restarted GMRES(m)). We introduce the notion of multi-Krylov method and show that AA(m) with general initial iterates {x 0 , . . . , xm} is a multi-Krylov method. We find that the AA(m) residual polynomials observe a periodic memory effect where increasing powers of the error iteration matrix M act on the initial residual as the iteration number increases. We derive several further results based on these polynomial residual update formulas, including orthogonality relations, a lower bound on the AA(1) acceleration coefficient β k , and explicit nonlinear recursions for the AA(1) residuals and residual polynomials that do not include the acceleration coefficient β k . We apply these results to study the influence of the initial guess on the asymptotic convergence factor of AA(1).

show abstract

Section: Introductionmentioning

confidence: 77%

“…In previous work, [11,13,16] have interpreted Nesterov acceleration, which is a form of AA(1) with a prescribed sequence of acceleration coefficients β k , as a Krylov method.…”

Section: Introductionmentioning

confidence: 99%

Anderson Acceleration as a Krylov Method with Application to Asymptotic Convergence Analysis

Sterck

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…We remark that Nesterov's acceleration strategy was first proposed in [29] to accelerate gradient type regularization method for linear as well as nonlinear illposed problems in Banach spaces and various numerical results were reported which demonstrate the striking performance; see also [27,28,35,39,45] for further numerical simulations. Although we have proved in Theorem 3.8 a convergence rate result for the method (3.18) under an a priori stopping rule, the regularization property of the method under the discrepancy principle is not yet established for general strongly convex R. However, when X is a Hilbert space and R(x) = x 2 /2, the regularization property of the corresponding method has been established in [35,39] based on a general acceleration framework in [21] using orthogonal polynomials; in particular it was observed in [35] that the parameter α plays an interesting role in deriving order optimal convergence rates. For an analysis of Nesterov's acceleration for nonlinear ill-posed problems in Hilbert spaces, one may refer to [28].…”

Section: 3mentioning

confidence: 92%

Convergence rates of a dual gradient method for constrained linear ill-posed problems

Jin

2022

Numer. Math.

View full text Add to dashboard Cite

In this paper we consider a dual gradient method for solving linear ill-posed problems $$Ax = y$$ A x = y , where $$A : X \rightarrow Y$$ A : X → Y is a bounded linear operator from a Banach space X to a Hilbert space Y. A strongly convex penalty function is used in the method to select a solution with desired feature. Under variational source conditions on the sought solution, convergence rates are derived when the method is terminated by either an a priori stopping rule or the discrepancy principle. We also consider an acceleration of the method as well as its various applications.

show abstract

“…In 2017, Neubauer considered the Nesterov scheme with a stopping rule (the discrepancy principle) and proved convergence rates under standard source conditions [39]. In 2021 Kindermann revisited the Nesterov scheme for linear ill-posed problems [30] and proved that this explicit two-point method is an optimal-order iterative regularization method. In 2017 Hubmer and Ramlau [26] considered the Nesterov scheme (with discrepancy principle) for nonlinear ill-posed problems (under the Scherzer condition [17, equation (11.6)]).…”

Section: Two-point Iterationsmentioning

confidence: 99%

On inertial iterated Tikhonov methods for solving ill-posed problems

Rabelo,

Leitão,

Madureira

2024

Inverse Problems

View full text Add to dashboard Cite

In this manuscript we propose and analyze an implicit two-point type method (or inertial method) for obtaining stable approximate solutions to linear ill-posed operator equations. The method is based on the iterated Tikhonov (iT) scheme. We establish convergence for exact data, and stability and semi-convergence for noisy data. Regarding numerical experiments we consider: i) a 2D Inverse Potential Problem, ii) an Image Deblurring Problem; the computational eﬃciency of the method is compared with standard implementations of the iT method.

show abstract

Optimal-order convergence of Nesterov acceleration for linear ill-posed problems*

Cited by 11 publications

References 16 publications

Anderson Acceleration as a Krylov Method with Application to Asymptotic Convergence Analysis

Anderson Acceleration as a Krylov Method with Application to Asymptotic Convergence Analysis

Convergence rates of a dual gradient method for constrained linear ill-posed problems

On inertial iterated Tikhonov methods for solving ill-posed problems

Contact Info

Product

Resources

About