Optimal Solutions of Well-Posed Linear Systems via Low-Precision Right-Preconditioned GMRES with Forward and Backward Stabilization

Abstract

In scientific applications, linear systems are typically well-posed, and yet the coefficient matrices may be nearly singular in that the condition number κ(A) may be close to 1/εw, where εw denotes the unit roundoff of the working precision. Accurate and efficient solutions to such systems pose daunting challenges. It is well known that iterative refinement (IR) can make the forward error independent of κ(A) if κ(A) is sufficiently smaller than 1/εw and the residual is computed in higher precision. Recently, Carson and Higham [SISC, 39(6), 2017] proposed a variant of IR called GMRES-IR, which replaced the triangular solves in IR with left-preconditioned GMRES using the LU factorization of A as the preconditioner. GMRES-IR relaxed the requirement on κ(A) in IR, but it requires triangular solves to be evaluated in higher precision, complicating its application to large-scale sparse systems. We propose a new iterative method, called Forward-and-Backward Stabilized Minimal Residual or FBSMR, by conceptually hybridizing right-preconditioned GMRES (RP-GMRES) with quasi-minimization. We develop FBSMR based on a new theoretical framework of essential-forward-and-backward stability (EFBS ), which extends the backward error analysis to consider the intrinsic condition number of a well-posed problem. We stabilize the forward and backward errors in RP-GMRES to achieve EFBS by evaluating a small portion of the algorithm in higher precision while evaluating the preconditioner in lower precision. FBSMR can achieve optimal accuracy in terms of both forward and backward errors for well-posed problems with unpolluted matrices, independently of κ(A). With low-precision preconditioning, FBSMR can reduce the computational, memory, and energy requirements over direct methods with or without IR. FBSMR can also leverage parallelization-friendly classical Gram-Schmidt in Arnoldi iterations without compromising EFBS. We demonstrate the effectiveness of FBSMR using both random and realistic linear systems.