Preconditioning Linear Least-Squares Problems by Identifying a Basis Matrix

Abstract

Abstract. We study the solution of the linear least-squares problem minx b − Ax 2 2 where the matrix A ∈ R m×n (m ≥ n) has rank n and is large and sparse. We assume that A is available as a matrix, not an operator. The preconditioning of this problem is difficult because the matrix A does not have the properties of differential problems that make standard preconditioners effective. Incomplete Cholesky techniques applied to the normal equations do not produce a well-conditioned problem. We attempt to bypass the ill-conditioning by finding an n × n nonsingular submatrix B of A that reduces the Euclidean norm of AB −1 . We use B to precondition a symmetric quasi-definite linear system whose condition number is then independent of the condition number of A and has the same solution as the original least-squares problem. We illustrate the performance of our approach on some standard test problems and show it is competitive with other approaches.

Permuted Graph Matrices and Their Applications

“…We sketch two applications here, taken from [48] and [3], respectively. More examples will appear in the next sections.…”

“…to f P , starting from X, obtaining a new point X 0 ; if kX 0 k max > then Use Algorithm 1 to find a permuted graph representation P 00 G .X 00 / of P G .X 0 /, with threshold ; replace .X; P / with .X 00 ; P 00 / and continue; else replace X with X 0 and continue; end end A different context in optimization in which suitable permutations and graph forms have appeared recently is the preconditioning and solution of large-scale saddle-point problems [3,15,16]. We present here the preconditioner for leastsquares problems appearing in [3]. A least-squares problem min x2C m kUx bk, for U 2 C .mCn/ m , can be reformulated as the augmented system…”

“…In practice, the matrix Q above is applied as a preconditioner; hence, to get faster convergence of preconditioned iterative methods, it is useful to choose a permuted graph basis with a small Ä.P G .X //. The authors in [3] suggest useful heuristic methods to find one for a large and sparse U .…”

Iterative Solution of Sparse Linear Least Squares using LU Factorization

“…There are also approaches based on LU factorization preconditioning [1,4,5,21]. LU preconditioning has been also studied in [14] with the objective of exploiting the recent progress in sparse LU factorization.…”

“…In [1], a partition A = A1 A2 is also used (where A 1 is a set of basis obtained with an LU factorization), and the augmented system is transformed into an equivalent symmetric quasi-definite system.…”