Roundoff error analysis of algorithms based on Krylov subspace methods

Arioli, Mario; Fassino, Claudia

doi:10.1007/bf01731978

Cited by 14 publications

(11 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As noticed in [32] and used in [7] (see also [3]), the Arnoldi process (2.2) with (2.1) ideally gives the QR factorization of [r 0 , AV k ], since on defining upper triangular…”

Section: Delayed Convergence Of Gmresmentioning

confidence: 97%

Residual and Backward Error Bounds in Minimum Residual Krylov Subspace Methods

Paige¹,

Strakoš²

2002

SIAM J. Sci. Comput.

View full text Add to dashboard Cite

Paige and Z. Strakoš, Bounds for the least squares distance using scaled total least squares, Numer. Math., to appear] revealing upper and lower bounds on the residual norm of any linear least squares (LS) problem were derived in terms of the total least squares (TLS) correction of the corresponding scaled TLS problem. In this paper theoretical results of [C. Paige and Z. Strakoš, Bounds for the least squares distance using scaled total least squares, Numer. Math., to appear] are extended to the GMRES context. The bounds that are developed are important in theory, but they also have fundamental practical implications for the finite precision behavior of the modified Gram-Schmidt implementation of GMRES, and perhaps for other minimum norm methods.

show abstract

“…As noticed in [32] and used in [7] (see also [3]), the Arnoldi process (2.2) with (2.1) ideally gives the QR factorization of [r 0 , AV k ], since on defining upper triangular…”

Section: Delayed Convergence Of Gmresmentioning

confidence: 97%

Residual and Backward Error Bounds in Minimum Residual Krylov Subspace Methods

Paige¹,

Strakoš²

2002

SIAM J. Sci. Comput.

View full text Add to dashboard Cite

show abstract

“…[25]. It was used in [8] and [1], and in particular in [18], in which we outlined another possible approach to backward stability analysis of MGS-GMRES. Here we have chosen a different way of proving the backward stability result, and this follows the spirit of [5] and [10].…”

Section: Mgs Applied Tomentioning

confidence: 99%

Modified Gram-Schmidt (MGS), Least Squares, and Backward Stability of MGS-GMRES

Paige¹,

Rozložńık²,

Strakoš³

2006

SIAM J. Matrix Anal. & Appl.

115

View full text Add to dashboard Cite

Abstract. The generalized minimum residual method (GMRES) [Y. Saad and M. Schultz, SIAM J. Sci. Statist. Comput., 7 (1986), pp. 856-869] for solving linear systems Ax = b is implemented as a sequence of least squares problems involving Krylov subspaces of increasing dimensions. The most usual implementation is modified Gram-Schmidt GMRES (MGS-GMRES). Here we show that MGS-GMRES is backward stable. The result depends on a more general result on the backward stability of a variant of the MGS algorithm applied to solving a linear least squares problem, and uses other new results on MGS and its loss of orthogonality, together with an important but neglected condition number, and a relation between residual norms and certain singular values.Key words. rounding error analysis, backward stability, linear equations, condition numbers, large sparse matrices, iterative solution, Krylov subspace methods, Arnoldi method, generalized minimum residual method, modified Gram-Schmidt, QR factorization, loss of orthogonality, least squares, singular values AMS subject classifications. 65F10, 65F20, 65F25, 65F35, 65F50, 65G50, 15A12, 15A42 DOI. 10.1137/050630416 1. Introduction. Consider a system of linear algebraic equations Ax = b, where A is a given n × n (unsymmetric) nonsingular matrix and b a nonzero n-dimensional vector. Given an initial approximation x 0 , one approach to finding x is to first compute the initial residual r 0 = b − Ax 0 . Using this, derive a sequence of Krylov subspaces. . , in some way, and look for approximate solutions x k ∈ x 0 + K k (A, r 0 ) . Various principles are used for constructing x k , which determine various Krylov subspace methods for solving Ax = b. Similarly, Krylov subspaces for A can be used to obtain eigenvalue approximations or to solve other problems involving A.Krylov subspace methods are useful for solving problems involving very large sparse matrices, since these methods use these matrices only for multiplying vectors, and the resulting Krylov subspaces frequently exhibit good approximation properties. The Arnoldi method [2] is a Krylov subspace method designed for solving the eigenproblem of unsymmetric matrices. The generalized minimum residual method (GMRES) [20] uses the Arnoldi iteration and adapts it for solving the linear system Ax = b. GMRES can be computationally more expensive per step than some other methods; see, for example, Bi-CGSTAB [24] and QMR [9] for unsymmetric A, and LSQR [16] for unsymmetric or rectangular A. However, GMRES is widely used for solving linear systems arising from discretization of partial differential equations, and

show abstract

“…Although out of the scope of this paper, one could potentially use existing results to prove that our approach will work with other Krylov subspace methods besides GMRES. One such potential method is the full orthogonalization method (FOM), for which bounds on the forward error have been given in [4].…”

mentioning

confidence: 99%

A New Analysis of Iterative Refinement and Its Application to Accurate Solution of Ill-Conditioned Sparse Linear Systems

Carson¹,

Higham

2017

SIAM J. Sci. Comput.

126

View full text Add to dashboard Cite

Abstract. Iterative refinement is a long-standing technique for improving the accuracy of a computed solution to a nonsingular linear system Ax = b obtained via LU factorization. It makes use of residuals computed in extra precision, typically at twice the working precision, and existing results guarantee convergence if the matrix A has condition number safely less than the reciprocal of the unit roundoff, u. We identify a mechanism that allows iterative refinement to produce solutions with normwise relative error of order u to systems with condition numbers of order u −1 or larger, provided that the update equation is solved with a relative error sufficiently less than 1. A new rounding error analysis is given, and its implications are analyzed. Building on the analysis, we develop a GMRES (generalized minimal residual)-based iterative refinement method (GMRES-IR) that makes use of the computed LU factors as preconditioners. GMRES-IR exploits the fact that even if A is extremely ill conditioned the LU factors contain enough information that preconditioning can greatly reduce the condition number of A. Our rounding error analysis and numerical experiments show that GMRES-IR can succeed where standard refinement fails, and that it can provide accurate solutions to systems with condition numbers of order u −1 and greater. Indeed, in our experiments with such matrices-both random and from the University of Florida Sparse Matrix Collection-GMRES-IR yields a normwise relative error of order u in at most three steps in every case.

show abstract

Roundoff error analysis of algorithms based on Krylov subspace methods

Cited by 14 publications

References 12 publications

Residual and Backward Error Bounds in Minimum Residual Krylov Subspace Methods

Residual and Backward Error Bounds in Minimum Residual Krylov Subspace Methods

Modified Gram-Schmidt (MGS), Least Squares, and Backward Stability of MGS-GMRES

A New Analysis of Iterative Refinement and Its Application to Accurate Solution of Ill-Conditioned Sparse Linear Systems

Contact Info

Product

Resources

About