Non-stationary Anderson acceleration with optimized damping

Chen, Kewang; Vuik, C.

doi:10.48550/arxiv.2202.05295

Cited by 7 publications

(10 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One is choosing different damping factors in each iteration, see our recent work on the non-stationary Anderson acceleration algorithm with optimized damping (AAoptD). 29 The other way of making AA to be a nonstationary algorithm is to alternate the window size during iterations. But no systematic ways have been proposed to dynamically alternate the window size .…”

Section: End Formentioning

confidence: 99%

See 1 more Smart Citation

Composite Anderson acceleration method with dynamic window-sizes and optimized damping

Chen¹,

Vuik²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper, we propose and analyze a set of fully non-stationary Anderson acceleration algorithms with dynamic window sizes and optimized damping. Although Anderson acceleration (AA) has been used for decades to speed up nonlinear solvers in many applications, most authors are simply using and analyzing the stationary version of Anderson acceleration (sAA) with fixed window size and a constant damping factor. The behavior and potential of the non-stationary version of Anderson acceleration methods remain an open question. Since most efficient linear solvers use composable algorithmic components. Similar ideas can be used for AA to solve nonlinear systems. Thus in the present work, to develop non-stationary Anderson acceleration algorithms, we first propose two systematic ways to dynamically alternate the window size by composition. One simple way to package sAA(m) with sAA(n) in each iteration is applying sAA(m) and sAA(n) separately and then average their results. It is an additive composite combination. The other more important way is the multiplicative composite combination, which means we apply sAA(m) in the outer loop and apply sAA(n) in the inner loop. By doing this, significant gains can be achieved. Secondly, to make AA to be a fully non-stationary algorithm, we need to combine these strategies with our recent work on the non-stationary Anderson acceleration algorithm with optimized damping (AAoptD), which is another important direction of producing non-stationary AA and nice performance gains have been observed. Moreover, we also investigate the rate of convergence of these nonstationary AA methods under suitable assumptions. Finally, our numerical results show that some of these proposed non-stationary Anderson acceleration algorithms converge faster than the stationary sAA method and they may significantly reduce the storage and time to find the solution in many cases.

show abstract

Section: End Formentioning

confidence: 99%

“…For more details on the implementation of ( ) and its performance, we refer the readers to our recent paper. 29…”

Section: Non-stationary Aa With Optimized Dampingmentioning

confidence: 99%

Composite Anderson acceleration method with dynamic window-sizes and optimized damping

Chen¹,

Vuik²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…For more details on the implementation of AAoptD(m) and its performance, we refer the readers to our recent paper. 31…”

Section: Nonstationary Aa With Optimized Dampingmentioning

confidence: 99%

“…One is choosing different damping factors 𝛽 k in each iteration, see our recent work on the nonstationary AA algorithm with optimized damping (AAoptD). 31 The other way of making AA to be a nonstationary algorithm is to alternate the window size during iterations. But no systematic ways have been proposed to dynamically alternate the window size m. Since most efficient linear solvers use composable algorithmic components, 32,33 similar ideas can be used for AA(m) and AA(n) to solve nonlinear systems.…”

Section: Algorithm 1 Anderson Acceleration: Aa(m)mentioning

confidence: 99%

Composite Anderson acceleration method with two window sizes and optimized damping

Chen

Vuik

2022

Numerical Meth Engineering

Self Cite

View full text Add to dashboard Cite

In this article, we propose and analyze a set of fully nonstationary Anderson acceleration (AA) algorithms with two window sizes and optimized damping. Although AA has been used for decades to speed up nonlinear solvers in many applications, most authors are simply using and analyzing the stationary version of AA (sAA) with fixed window size and a constant damping factor. The behavior and potential of the nonstationary version of AA methods remain an open question. Most efficient linear solvers however use composable algorithmic components. Similar ideas can be used for AA to solve nonlinear systems. Thus in the present work, to develop nonstationary AA algorithms, we firstpropose a systematic way to dynamically alternate the window size m by the multiplicative composite combination, which means we apply sAA(m) in the outer loop and apply sAA(n) in the inner loop. By doing this, significant gains can be achieved. Second, to make AA to be a fully nonstationary algorithm, we need to combine these strategies with our recent work on the nonstationary AA algorithm with optimized damping (AAoptD), which is another important direction of producing nonstationary AA and nice performance gains have been observed. Moreover, we also investigate the rate of convergence of these nonstationary AA methods under suitable assumptions. Finally, our numerical results show that some of these proposed nonstationary AA algorithms converge faster than the stationary sAA method and they may significantly reduce the storage and time to find the solution in many cases.

show abstract

“…where θ values can be found trough solving a minimization problem, which following the steps in [27,33,34] can be optimally reformulated for implementation as an unconstrained linear least squared problem :…”

Section: Anderson Accelerationmentioning

confidence: 99%

Fast image reverse filters through fixed point and gradient descent acceleration

Galetto¹,

Deng²

2022

Preprint

View full text Add to dashboard Cite

In this paper, we study the problem of reverse image filtering. An image filter denoted g(.), which is available as a black box, produces an observation b = g(x) when provided with an input x. The problem is to estimate the original input signal x from the black box filter g(x) and the observation b. We study and re-develop state-of-the-art methods from two points of view, fixed point iteration and gradient descent. We also explore the application of acceleration techniques for the two types of iterations. Through extensive experiments and comparison, we show that acceleration methods for both fixed point iteration and gradient descent help to speed up the convergence of state-of-the-art methods.

show abstract

Non-stationary Anderson acceleration with optimized damping

Cited by 7 publications

References 26 publications

Composite Anderson acceleration method with dynamic window-sizes and optimized damping

Composite Anderson acceleration method with dynamic window-sizes and optimized damping

Composite Anderson acceleration method with two window sizes and optimized damping

Fast image reverse filters through fixed point and gradient descent acceleration

Contact Info

Product

Resources

About