Three-Precision GMRES-Based Iterative Refinement for Least Squares Problems

Abstract

With the recent emergence of mixed precision hardware, there has been a renewed interest in its use for solving numerical linear algebra problems fast and accurately. The solution of least squares (LS) problems min x ∥b − Ax∥ 2 , where A ∈ R m×n , arise in numerous application areas. Overdetermined standard least squares problems can be solved by using mixed precision within the iterative refinement method of Björck, which transforms the least squares problem into an (m + n) × (m + n) "augmented" system. It has recently been shown that mixed precision GMRESbased iterative refinement can also be used, in an approach termed GMRES-LSIR. In practice, we often encounter types of least squares problems beyond standard least squares, including weighted least squares (WLS), min x ∥D 1/2 (b − Ax)∥ 2 , where D 1/2 is a diagonal matrix of weights. In this paper, we discuss a mixed precision FGMRES-WLSIR algorithm for solving WLS problems using two different preconditioners.

A survey of numerical linear algebra methods utilizing mixed-precision arithmetic

“…Carson et al (2020) show that the three-precision iterative refinement approach of Carson and Higham (2018) can be applied in this case; the theorems developed in Carson and Higham (2018) for the forward error and normwise and componentwise backward error for iterative refinement of linear systems are applicable. The only thing that must change is the analysis of the method for solving the correction equation, since we now work with a QR factorization of A , which can be used in various ways.…”

“…The work in Carson et al (2020) also extends the GMRES-based refinement scheme of Carson and Higham (2017) to the least squares case and shows that one can construct a left preconditioner using the existing QR factors of A such that GMRES provably converges to a backward stable solution of the preconditioned augmented system. Further, it is shown that an existing preconditioner developed for saddle point systems can also work well in the GMRES-based approach in practice, even though the error analysis is not applicable.…”

“…For more details on scaling see Higham et al (2019), Higham and Pranesh (2021), and Carson et al (2020).…”

“…As mentioned, the Cholesky-based approach described in the previous section is intended only for the case where the matrix is very well conditioned. Another approach to mixed-precision least-squares iterative refinement with a less stringent requirement on the condition number is presented by Carson et al (2020). This approach is based on using the QR factorization:…”

“…Further, it is shown that an existing preconditioner developed for saddle point systems can also work well in the GMRES-based approach in practice, even though the error analysis is not applicable. We refer the reader to Carson et al (2020) for further details.…”