A proof that Anderson acceleration improves the convergence rate in linearly converging fixed point methods (but not in those converging quadratically)

Evans, Claire; Pollock, Sara; Rebholz, Leo G.; Xiao, Mengying

doi:10.48550/arxiv.1810.08455

Cited by 2 publications

(2 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our experiments show that Anderson acceleration is effective in reducing the number of iterations, but we do not have a theoretical guarantee for such property. This is still an open research problem, and the only existing result we are aware of is [Evans et al 2018], which proves that Anderson acceleration improves the convergence rate for linearly converging fixed-point methods if a set of strong assumptions is satisfied. Further theoretical analysis of our method is needed to understand and guarantee its performance.…”

Section: Discussionmentioning

confidence: 99%

Accelerating ADMM for efficient simulation and optimization

et al. 2019

View full text Add to dashboard Cite

show abstract

Section: Discussionmentioning

confidence: 99%

Accelerating ADMM for efficient simulation and optimization

et al. 2019

View full text Add to dashboard Cite

show abstract

“…It is closely related to Pulay mixing [28] and DIIS (direct inversion on the iterative subspace) [17,29], which are prominent methods in self-consistent field theory [5,7]. AA is also becoming popular in the numerical analysis community [35,33,11,38,25]. The literature on this subject is broad, so we only mention a few papers to show the variety of results obtained by AA.…”

Section: Introductionmentioning

confidence: 99%

Anderson Acceleration for Seismic Inversion

Yang

2020

Preprint

View full text Add to dashboard Cite

The state-of-art seismic imaging techniques treat inversion tasks such as FWI and LSRTM as PDE-constrained optimization problems. Due to the large-scale nature, gradient-based optimization algorithms are preferred in practice to update the model iteratively. Higher-order methods converge in fewer iterations but often require higher computational costs, more line search steps, and bigger memory storage. A balance among these aspects has to be considered. We propose using Anderson acceleration (AA), a popular strategy to speed up the convergence of fixed-point iterations, to accelerate the steepest descent algorithm, which we innovatively treat as a fixed-point iteration. Independent of the dimensionality of the unknown parameters, the computational cost of implementing the method can be reduced to an extremely low-dimensional least-squares problem. The cost can be further reduced by a low-rank update. We discuss the theoretical connections and the differences between AA and other well-known optimization methods such as L-BFGS and the restarted GMRES and compare their computational cost and memory demand. Numerical examples of FWI and LSRTM applied to the Marmousi benchmark demonstrate the acceleration effects of AA. Compared with the steepest descent method, AA can achieve fast convergence and provide competitive results with some quasi-Newton methods, making it an attractive optimization strategy for seismic inversion.

show abstract

A proof that Anderson acceleration improves the convergence rate in linearly converging fixed point methods (but not in those converging quadratically)

Cited by 2 publications

References 13 publications

Accelerating ADMM for efficient simulation and optimization

Accelerating ADMM for efficient simulation and optimization

Anderson Acceleration for Seismic Inversion

Contact Info

Product

Resources

About