Residual and Backward Error Bounds in Minimum Residual Krylov Subspace Methods

Paige, Christopher C.; Strakoš, Zdeněk

doi:10.1137/s1064827500381239

Cited by 30 publications

(47 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Section 6 introduces the key step used to prove convergence of these iterations. In section 7.1 we prove the backward stability of the MGS algorithm applied to solving linear least squares problems of the form required by the MGS-GMRES algorithm, and in section 7.2 we show how loss of orthogonality is directly related to new normwise relative backward errors of a sequence of different least squares problems, supporting a conjecture on the convergence of MGS-GMRES and its loss of orthogonality; see [18]. In section 8.1 we show that at every step MGS-GMRES computes a backward stable solution for that step's linear least squares problem, and in section 8.2 we show that one of these solutions is also a backward stable solution for (1.1) in at most n+1 MGS steps.…”

mentioning

confidence: 68%

See 1 more Smart Citation

Modified Gram-Schmidt (MGS), Least Squares, and Backward Stability of MGS-GMRES

Paige¹,

Rozložńık²,

Strakoš³

2006

SIAM J. Matrix Anal. & Appl.

Self Cite

115

View full text Add to dashboard Cite

Abstract. The generalized minimum residual method (GMRES) [Y. Saad and M. Schultz, SIAM J. Sci. Statist. Comput., 7 (1986), pp. 856-869] for solving linear systems Ax = b is implemented as a sequence of least squares problems involving Krylov subspaces of increasing dimensions. The most usual implementation is modified Gram-Schmidt GMRES (MGS-GMRES). Here we show that MGS-GMRES is backward stable. The result depends on a more general result on the backward stability of a variant of the MGS algorithm applied to solving a linear least squares problem, and uses other new results on MGS and its loss of orthogonality, together with an important but neglected condition number, and a relation between residual norms and certain singular values.Key words. rounding error analysis, backward stability, linear equations, condition numbers, large sparse matrices, iterative solution, Krylov subspace methods, Arnoldi method, generalized minimum residual method, modified Gram-Schmidt, QR factorization, loss of orthogonality, least squares, singular values AMS subject classifications. 65F10, 65F20, 65F25, 65F35, 65F50, 65G50, 15A12, 15A42 DOI. 10.1137/050630416 1. Introduction. Consider a system of linear algebraic equations Ax = b, where A is a given n × n (unsymmetric) nonsingular matrix and b a nonzero n-dimensional vector. Given an initial approximation x 0 , one approach to finding x is to first compute the initial residual r 0 = b − Ax 0 . Using this, derive a sequence of Krylov subspaces. . , in some way, and look for approximate solutions x k ∈ x 0 + K k (A, r 0 ) . Various principles are used for constructing x k , which determine various Krylov subspace methods for solving Ax = b. Similarly, Krylov subspaces for A can be used to obtain eigenvalue approximations or to solve other problems involving A.Krylov subspace methods are useful for solving problems involving very large sparse matrices, since these methods use these matrices only for multiplying vectors, and the resulting Krylov subspaces frequently exhibit good approximation properties. The Arnoldi method [2] is a Krylov subspace method designed for solving the eigenproblem of unsymmetric matrices. The generalized minimum residual method (GMRES) [20] uses the Arnoldi iteration and adapts it for solving the linear system Ax = b. GMRES can be computationally more expensive per step than some other methods; see, for example, Bi-CGSTAB [24] and QMR [9] for unsymmetric A, and LSQR [16] for unsymmetric or rectangular A. However, GMRES is widely used for solving linear systems arising from discretization of partial differential equations, and

show abstract

mentioning

confidence: 68%

“…[25]. It was used in [8] and [1], and in particular in [18], in which we outlined another possible approach to backward stability analysis of MGS-GMRES. Here we have chosen a different way of proving the backward stability result, and this follows the spirit of [5] and [10].…”

Section: Mgs Applied Tomentioning

confidence: 99%

Modified Gram-Schmidt (MGS), Least Squares, and Backward Stability of MGS-GMRES

Paige¹,

Rozložńık²,

Strakoš³

2006

SIAM J. Matrix Anal. & Appl.

Self Cite

115

View full text Add to dashboard Cite

show abstract

“…While (17) was in [8, Corollary 5.1] derived from (9), it is not always true that the latter is tighter. When δ(γ) ≈ 1 and r ≈ b , it is possible for the upper bound in (17) to be smaller than that in (9). But in this case…”

Section: Corollary 2 Under the Same Conditions And Assumptions As In mentioning

confidence: 96%

“…The results presented here have been successfully applied outside the Errors-in-Variables Modeling field for analysis of convergence and numerical stability of Krylov subspace methods, see [9], [6].…”

Section: Tightness Parametermentioning

confidence: 99%

Bounds for the Least Squares Residual Using Scaled Total Least Squares

Paige

Strakoš

2002

Total Least Squares and Errors-in-Variables Modeling

Self Cite

View full text Add to dashboard Cite

The standard approaches to solving overdetermined linear systems Ax ≈ b construct minimal corrections to the data to make the corrected system compatible. In ordinary least squares (LS) the correction is restricted to the right hand side b, while in scaled total least squares (Scaled TLS) [10; 7] corrections to both b and A are allowed, and their relative sizes are determined by a real positive parameter γ. As γ → 0, the Scaled TLS solution approaches the LS solution. Fundamentals of the Scaled TLS problem are analyzed in our paper [7] and in the contribution in this book entitled Unifying least squares, total least squares and data least squares. This contribution is based on the paper [8]. It presents a theoretical analysis of the relationship between the sizes of the LS and Scaled TLS corrections (called the LS and Scaled TLS distances) in terms of γ. We give new upper and lower bounds on the LS distance in terms of the Scaled TLS distance, compare these to existing bounds, and examine the tightness of the new bounds.This work can be applied to the analysis of iterative methods which minimize the residual norm [9; 6].

show abstract

“…which is the default criterion in ILUPACK [11], based on the analysis in [3]; see also [33]. Since the results were very similar and the full test can not be done at each iteration we only show results using the relative residual stopping criterion (6.1).…”

Section: Choosing Parametersmentioning

confidence: 99%

Extensions of Certain Graph-based Algorithms for Preconditioning

Fritzsche¹,

Frommer²,

Szyld³

2007

SIAM J. Sci. Comput.

View full text Add to dashboard Cite

Abstract. The original TPABLO algorithms are a collection of algorithms which compute a symmetric permutation of a linear system such that the permuted system has a relatively full block diagonal with relatively large nonzero entries. This block diagonal can then be used as a preconditioner. We propose and analyze three extensions of this approach: we incorporate a nonsymmetric permutation to obtain a large diagonal, we use a more general parametrization for TPABLO, and we use a block Gauss-Seidel preconditioner which can be implemented to have the same execution time as the corresponding block Jacobi preconditioner. Experiments are presented showing that for certain classes of matrices, the block Gauss-Seidel preconditioner used with the system permuted with the new algorithm can outperform the best ILUT preconditioners in a large set of experiments.

show abstract

Residual and Backward Error Bounds in Minimum Residual Krylov Subspace Methods

Cited by 30 publications

References 26 publications

Modified Gram-Schmidt (MGS), Least Squares, and Backward Stability of MGS-GMRES

Modified Gram-Schmidt (MGS), Least Squares, and Backward Stability of MGS-GMRES

Bounds for the Least Squares Residual Using Scaled Total Least Squares

Extensions of Certain Graph-based Algorithms for Preconditioning

Contact Info

Product

Resources

About