Accurate conjugate gradient methods for families of shifted systems

Abstract

We consider the solution of the linear systemfor various real values of σ. This family of shifted systems arises, for example, in Tikhonov regularization and computations in lattice quantum chromodynamics. For each single shift σ this system can be solved using the conjugate gradient method for least squares problems (CGLS). In literature various implementations of the, so-called, multishift CGLS methods have been proposed. These methods are mathematically equivalent to applying the CGLS method to each shifted system separately but they solve all systems simultaneously and require only two matrix-vector products (one by A and one by A T ) and two inner products per iteration step. Unfortunately, numerical experiments show that, due to roundoff errors, in some cases these implementations of the multishift CGLS method can only attain an accuracy that depends on the square of condition number of the matrix A. In this paper we will argue that, in the multishift CGLS method, the impact on the attainable accuracy of rounding errors in the Lanczos part of the method is independent of the effect of roundoff errors made in the construction of the iterates. By making suitable design choices for both parts, we derive a new (and efficient) implementation that tries to remove the limitation of previous proposals. A partial roundoff error analysis and various numerical experiments show promising results.

Robust reconstruction of B1+ maps by projection into a spherical functions space

“…In conclusion, the fitting procedure is: Step 1: Solve c 0 = arg min c ‖ F r c − d ‖ by multishift CGLS algorithm Step 2: Solve c 1 = arg min c ‖| F r c | − d ‖ by a “pattern search” algorithm, with initial guess c 0 and boundaries ‖ c ‖ ∞ ≤ 2 ‖ c 0 ‖ ∞ . Step 3: Solve ĉ = arg min c ‖| F r c | − d ‖ by a “line search” algorithm, with initial guess c 1 and boundaries as in Step 2.…”

Beyond smooth inversion: the use of nullspace projection for the exploration of non-uniqueness in MT

“…Most researchers prefer simpler heuristic methods, however, because the aforementioned methods require the knowledge of error structure as (1), or the solution of many forward problems (2, 3), rendering the method computationally expensive. There are possible remedies for this, for example, the transformation of the linear equations to shifted systems, which can be solved very effectively for many τ (van den Eshof & Sleijpen 2004; Frommer & Maass 1999). Among the heuristic methods for choosing τ there are cooling schedules (e.g.…”

Sparse Least-Squares Methods in the Parallel Machine Learning (PML) Framework

“…Our parallel implementation in PML is therefore based on the CGLS algorithm for the multi-shift case, as described in [9] and further evaluated in [24]. The most expensive operations algorithm, both in terms of the computational and I/O costs, are the matrix-vector multiplications involving X and X T in each iteration of the Lanczos procedure, which are parallelized by partitioning the rows of X among the processors, and computing the required matrix-vector products in a distributed fashion, with the final result being collected in the designated master processor.…”