GMRES algorithms over 35 years

Zou, Qinmeng

doi:10.48550/arxiv.2110.04017

Cited by 4 publications

(8 citation statements)

References 275 publications

(418 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Let us now characterize the quasioptimality of such a projection when Q and H were obtained with the RBGS-Arnoldi algorithm. Under the stability conditions of RBGS, the Arnoldi identity (22) implies that…”

Section: Linear Systems: Randomized Block Gmres Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Randomized block Gram-Schmidt process for solution of linear systems and eigenvalue problems

Balabanov¹,

Grigori²

2021

Preprint

View full text Add to dashboard Cite

We propose a block version of the randomized Gram-Schmidt process for computing a QR factorization of a matrix. Our algorithm inherits the major properties of its single-vector analogue from [Balabanov and Grigori, 2020] such as higher efficiency than the classical Gram-Schmidt algorithm and stability of the modified Gram-Schmidt algorithm, which can be refined even further by using multi-precision arithmetic. As in [Balabanov and Grigori, 2020], our algorithm has an advantage of performing standard high-dimensional operations, that define the overall computational cost, with a unit roundoff independent of the dominant dimension of the matrix. This unique feature makes the methodology especially useful for large-scale problems computed on low-precision arithmetic architectures.Block algorithms are advantageous in terms of performance as they are mainly based on cache-friendly matrix-wise operations, and can reduce communication cost in high-performance computing. The block Gram-Schmidt orthogonalization is the key element in the block Arnoldi procedure for the construction of Krylov basis, which in its turn is used in GMRES and Rayleigh-Ritz methods for the solution of linear systems and clustered eigenvalue problems. In this article, we develop randomized versions of these methods, based on the proposed randomized Gram-Schmidt algorithm, and validate them on nontrivial numerical examples.

show abstract

Section: Linear Systems: Randomized Block Gmres Methodsmentioning

confidence: 99%

“…It is also used in s-step, enlarged and other communication-avoiding Krylov subspace methods [12,14]. Please see [9] and the references therein for an extensive overview of BGS variants, and [2,17,18,22] for the underlying block Krylov methods.…”

Section: Introductionmentioning

confidence: 99%

Randomized block Gram-Schmidt process for solution of linear systems and eigenvalue problems

Balabanov¹,

Grigori²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…R k,k = R (2) k,k R (1) k,k 11: end for Barlow and Smoktunowicz give a detailed proof and show that if…”

Section: Block Classical Gram-schmidt With Reorthogonalizationmentioning

confidence: 99%

“…BCGS is a natural generalization of the classical Gram-Schmidt (CGS) method and plays an important role in the s-step GMRES algorithm. 1,2 It replaces matrix-vector products with matrix-matrix products, thus allowing BLAS-3 cache performance. Despite this promising property, BCGS inherits the numerical instability of CGS.…”

Section: Introductionmentioning

confidence: 99%

A flexible block classical Gram–Schmidt skeleton with reorthogonalization

Zou

2023

Numerical Linear Algebra App

View full text Add to dashboard Cite

We investigate a variant of the reorthogonalized block classical Gram–Schmidt method for computing the QR factorization of a full column rank matrix. Our aim is to bound the loss of orthogonality even when the first local QR algorithm is only conditionally stable. In particular, this allows the use of modified Gram–Schmidt instead of Householder transformations as the first local QR algorithm. Numerical experiments confirm the stable behavior of the new variant. We also examine the use of non‐QR local factorization and show by example that the resulting variants, although less stable, may also be applied to ill‐conditioned problems.

show abstract

“…The GMRES algorithm of Saad and Schultz [9] has a thirty-five year history and alternative formulations of the basic algorithm have been proposed over that time frame. A comprehensive review of these is presented by Zou [22]. In particular, pipelined s-step and block algorithms have been proposed which are better able to hide latency in parallel implementations and are described in Yamazaki et al [23].…”

Section: Introductionmentioning

confidence: 99%

Iterated Gauss-Seidel GMRES

Thomas¹,

Carson²,

Rozložník³

et al. 2022

Preprint

View full text Add to dashboard Cite

The GMRES algorithm of Saad and Schultz (1986) for nonsymmetric linear systems relies on the Arnoldi expansion of the Krylov basis. The algorithm computes the QR factorization of the matrix B = [r 0 , AV k ] at each iteration. Despite an O(ε)κ(B) loss of orthogonality, the modified Gram-Schmidt (MGS) formulation was shown to be backward stable in the seminal papers by Paige, et al. (2006) and Paige and Strakoš (2002). Classical Gram-Schmidt (CGS) exhibits an O(ε)κ 2 (B) loss of orthogonality, whereas DCGS-2 (CGS with delayed reorthogonalization) reduces this to O(ε) in practice (without a formal proof). We present a post-modern (viz. not classical) GMRES algorithm based on Ruhe (1983) and the low-synch algorithms of Świrydowicz et al (2020) that achieves O(ε) Av k 2 /h k+1,k loss of orthogonality. By projecting the vector Av k , with Gauss-Seidel relaxation, onto the orthogonal complement of the space spanned by the computed Krylov vectors V k where V T k V k = I + L k + L T k , we can further demonstrate that the loss of orthogonality is at most O(ε)κ(B). For a broad class of matrices, unlike MGS-GMRES, significant loss of orthogonality does not occur and the relative residual no longer stagnates for highly non-normal systems. The Krylov vectors remain linearly independent and the smallest singular value of V k is not far from one. We also demonstrate that Henrici's departure from normality of the lower triangular matrixk is an appropriate quantity for detecting the loss of orthogonality. Our new algorithm results in an almost symmetric correction matrix T k .

show abstract

GMRES algorithms over 35 years

Cited by 4 publications

References 275 publications

Randomized block Gram-Schmidt process for solution of linear systems and eigenvalue problems

Randomized block Gram-Schmidt process for solution of linear systems and eigenvalue problems

A flexible block classical Gram–Schmidt skeleton with reorthogonalization

Iterated Gauss-Seidel GMRES

Contact Info

Product

Resources

About