Nearly Linear Time Algorithms for Preconditioning and Solving Symmetric, Diagonally Dominant Linear Systems

Abstract

Abstract. We present a randomized algorithm that on input a symmetric, weakly diagonally dominant n-by-n matrix A with m nonzero entries and an n-vector b produces anx such thatc n log(1/ )) for some constant c. By applying this algorithm inside the inverse power method, we compute approximate Fiedler vectors in a similar amount of time. The algorithm applies subgraph preconditioners in a recursive fashion. These preconditioners improve upon the subgraph preconditioners first introduced by Vaidya in 1990. For any symmetric, weakly diagonally dominant matrix A with nonpositive off-diagonal entries and k ≥ 1, we construct in time O(m log c n) a preconditioner B of A with at most 2(n − 1) + O((m/k) log 39 n) nonzero off-diagonal entries such that the finite generalized condition number κ f (A, B) is at most k, for some other constant c. In the special case when the nonzero structure of the matrix is planar the corresponding linear system solver runs in expected time O(n log 2 n + n log n log log n log(1/ )). We hope that our introduction of algorithms of low asymptotic complexity will lead to the development of algorithms that are also fast in practice. 1. Introduction. We design an algorithm with nearly optimal asymptotic complexity for solving linear systems in symmetric, weakly diagonally dominant (SDD 0 ) matrices. The algorithm applies a classical iterative solver, such as the preconditioned conjugate gradient or the preconditioned Chebyshev method, with a novel preconditioner that we construct and analyze using techniques from graph theory. Linear systems in these preconditioners may be reduced to systems of smaller size in linear time by use of a direct method. The smaller linear systems are solved recursively. The resulting algorithm solves linear systems in SDD 0 matrices in time that is asymptotically almost linear in their number of nonzero entries. Our analysis does not make any assumptions about the nonzero structure of the matrix and thus may be applied to the solution of the systems in SDD 0 matrices that arise in any application, such as the

Runtime guarantees for regression problems

“…Our algorithm is similar to convex optimization algorithms in that each iteration of it solves a quadratic minimization problem, which is equivalent to solving a linear system. The speedup over previous algorithms come from the existence of much faster solvers for graph related linear systems [38], although our approaches are also applicable to situations involving other underlying quadratic minimization problems.…”

Graphs, Vectors, and Matrices

“…The first is that it contains many beautiful theorems. The second is that we can compute good approximations of the eigenvalues and eigenvectors of a Laplacian matrix very quickly [ST14]. Thus, they provide computationally tractable analyses and can serve as the basis of many useful heuristics.…”

Dirichlet PageRank and Ranking Algorithms Based on Trust and Distrust

“…Since the matrix βI S + L S is diagonally dominant, it can be solved approximately in nearly-linear time with a SpielmanTeng Solver [22], but we will also give a simpler algorithm ApproxDirichPR to compute approximate Dirichlet PageRank vectors. This approximation algorithm is faster and has a better approximation ratio if the constant α is not too small.…”