Convergence Analysis for Anderson Acceleration

Abstract

Anderson(m) is a method for acceleration of fixed point iteration which stores m + 1 prior evaluations of the fixed point map and computes the new iteration as a linear combination of those evaluations. Anderson(0) is fixed point iteration. In this paper we show that Anderson(m) is locally r-linearly convergent if the fixed point map is a contraction and the coefficients in the linear combination remain bounded. We prove q-linear convergence of the residuals for linear problems for Anderson(1) without the assumption of boundedness of the coefficients. We observe that the optimization problem for the coefficients can be formulated and solved in non-standard ways and report on numerical experiments which illustrate the ideas.

Benchmarking results for the Newton–Anderson method

“…If the norm · used in the optimization step of Algorithm 3 is induced by an inner product ( ·, · ), Algorithm 3 reduces to Algorithm 2, for m = 1 (and reduces to Algorithm 1, for m = 0). See [13,14] for the equivalence of this form of the Anderson algorithm to that originally stated in [11], and [15] for a results on optimization in norms not induced by inner products. Throughout the remainder, the norm · will denote the l 2 norm over R n .…”

Anderson acceleration of the alternating projections method for computing the nearest correlation matrix

“…For nonlinear problems Rohwedder and Schneider [43] show that Anderson acceleration is locally linearly convergent under certain conditions. Adding to the above convergence analysis is the recent work by Toth and Kelley [46] concerning Anderson acceleration with m k = min(m, k), for a fixed m, applied to contractive mappings.…”

Convergence analysis of Anderson‐type acceleration of Richardson's iteration

“…Next, we prove a convergence result for AAR analogous to the one found theorem 2.1 in the work of Toth and Kelley for AR.…”

“…Similarly to what was already discussed in the work of Toth and Kelley for AR, requiring for AAR to converge is too restrictive and impractical in many cases. If the condition does not hold, truncated AAR cannot be guaranteed to converge in general.…”

“…In the present work, we study the mathematical properties of AAR in comparison with more standard techniques such as full GMRES, 1,12 truncated GMRES, 1 restarted GMRES, 1 and Anderson-Richardson. [13][14][15][16][17] All the convergence analyses and comparisons between algorithms conducted in this work assume exact arithmetic.…”