A fast randomized algorithm for the approximation of matrices

Woolfe, Franco; Liberty, Edo; Rokhlin, Vladimir; Tygert, Mark

doi:10.1016/j.acha.2007.12.002

Cited by 276 publications

(282 citation statements)

References 17 publications

(43 reference statements)

Supporting

Mentioning

276

Contrasting

Order By: Relevance

“…4, and is proven (in a slightly different form) as Lemma 4.4 of ref. 8. The lemma provides a highly probable upper bound on the condition number of the product of the l × m matrix G defined in Eq.…”

Section: Remark 1 Our Earlier Articles Refs 7 and 8 Omitted The Mmentioning

confidence: 99%

A fast randomized algorithm for overdetermined linear least-squares regression

Rokhlin

Tygert²

2008

Proc. Natl. Acad. Sci. U.S.A.

Self Cite

161

210

View full text Add to dashboard Cite

We introduce a randomized algorithm for overdetermined linear least-squares regression. Given an arbitrary full-rank m × n matrix A with m ≥ n, any m × 1 vector b, and any positive real number ε, the procedure computes an n × 1 vector x such that x minimizes the Euclidean norm A x − b to relative precision ε. The algorithm typically requires O((log(n) + log(1/ε))mn + n 3 ) floating-point operations. This cost is less than the O(mn 2 ) required by the classical schemes based on QR-decompositions or bidiagonalization. We present several numerical examples illustrating the performance of the algorithm.L east-squares fitting has permeated the sciences and engineering after its introduction over two centuries ago (see, for example, ref. 1 for a brief historical review). Linear least-squares regression is fundamental in the analysis of data, such as that generated from biology, econometrics, engineering, physics, and many other technical disciplines.Perhaps the most commonly encountered formulation of linear least-squares regression involves a full-rank m × n matrix A and an m × 1 column vector b, with m ≥ n; the task is to find an n × 1 column vector x such that the Euclidean norm Ax − b is minimized. Classical algorithms using QR-decompositions or bidiagonalization requirefloating-point operations in order to compute x (see, for example, ref. 1 or Chapter 5 in ref.2). The present article introduces a randomized algorithm that, given any positive real number ε, computes a vector x minimizing Ax − b to relative precision ε, that is, the algorithm produces a vector x such thatThis algorithm typically requiresoperations. When n is sufficiently large and m is much greater than n (that is, the regression is highly overdetermined), then the cost in Eq. 3 is less than the cost in Eq. 1. Furthermore, in the numerical experiments of Numerical Results, the algorithm of the present article runs substantially faster than the standard methods based on QR-decompositions. The method of the present article is an extension of the methods introduced in refs. 3-5. Their algorithms and ours have similar costs; however, for the computation of x minimizing Ax − b to relative precision ε, the earlier algorithms involve costs proportional to 1/ε, whereas the algorithm of the present paper involves a cost proportional to log(1/ε) (see Eq. 3 above).The present article describes algorithms optimized for the case when the entries of A and b are complex valued. Needless to say, real-valued versions of our schemes are similar. This article has the following structure: The first section sets the notation. The second section discusses a randomized linear transformation which can be applied rapidly to arbitrary vectors. The third section provides the relevant mathematical apparatus. The fourth section describes the algorithm of the present article. The fifth section illustrates the performance of the algorithm via several numerical examples. The sixth section draws conclusions and proposes directions for future work. NotationIn this section, we set ...

show abstract

Section: Remark 1 Our Earlier Articles Refs 7 and 8 Omitted The Mmentioning

confidence: 99%

A fast randomized algorithm for overdetermined linear least-squares regression

Rokhlin

Tygert²

2008

Proc. Natl. Acad. Sci. U.S.A.

Self Cite

161

210

View full text Add to dashboard Cite

show abstract

“…For a numerically low rank matrix, its approximate singular value decomposition can be efficiently evaluated using randomized algorithms (see e.g. [33,34,35]). The idea is briefly reviewed as below, though presented in a slightly non-standard way.…”

Section: Spectrum Sweeping Methods For Estimating Spectral Densitiesmentioning

confidence: 99%

“…It is easy to verify that such choice of h(t) satisfies Eq. (34) and is periodic on (−1, 1). There are many choice of π(t), and here we use (36) π…”

Section: Application To Trace Estimation Of General Matrix Functionsmentioning

confidence: 99%

Randomized estimation of spectral densities of large matrices made accurate

Lin

2016

Numer. Math.

View full text Add to dashboard Cite

Abstract. For a large Hermitian matrix A ∈ C N ×N , it is often the case that the only affordable operation is matrix-vector multiplication. In such case, randomized method is a powerful way to estimate the spectral density (or density of states) of A. However, randomized methods developed so far for estimating spectral densities only extract information from different random vectors independently, and the accuracy is therefore inherently limitedwhere Nv is the number of random vectors.In this paper we demonstrate that the "O(1/ √ Nv) barrier" can be overcome by taking advantage of the correlated information of random vectors when properly filtered by polynomials of A. Our method uses the fact that the estimation of the spectral density essentially requires the computation of the trace of a series of matrix functions that are numerically low rank. By repeatedly applying A to the same set of random vectors and taking different linear combination of the results, we can sweep through the entire spectrum of A by building such low rank decomposition at different parts of the spectrum. Under some assumptions, we demonstrate that a robust and efficient implementation of such spectrum sweeping method can compute the spectral density accurately with O(N 2 ) computational cost and O(N ) memory cost. Numerical results indicate that the new method can significantly outperform existing randomized methods in terms of accuracy. As an application, we demonstrate a way to accurately compute a trace of a smooth matrix function, by carefully balancing the smoothness of the integrand and the regularized density of states using a deconvolution procedure.

show abstract

“…As shown in [12], this cost can be lowered to O mn log(l) + lkn log(n) , where l is an integer greater than but close to k (in applications, l = k + 10 is typical.) The first term is the cost of obtaining l samples of the range of A via an accelerated fast Fourier transform.…”

Section: Hierarchical-block-separable Matricesmentioning

confidence: 99%

Efficient Preconditioning of hp-FEM Matrices by Hierarchical Low-Rank Approximations

Gatto

Hesthaven

2017

J Sci Comput

View full text Add to dashboard Cite

We introduce a preconditioner based on low-rank compression of Schur complements. The construction is inspired by standard nested-dissection and relies on the assumption that the Schur complements can be approximated to high precision by Hierarchical-Block-Separable matrices. We build the preconditioner as an approximate LDM t factorization of a given matrix A, and no knowledge of A in assembled form is required by the construction. The LDM t is amenable to fast inversion and the inverse can be applied fast as well. We investigate the behavior of the preconditioner in the context of DG finite element approximations of elliptic and hyperbolic problems.

show abstract

A fast randomized algorithm for the approximation of matrices

Cited by 276 publications

References 17 publications

A fast randomized algorithm for overdetermined linear least-squares regression

A fast randomized algorithm for overdetermined linear least-squares regression

Randomized estimation of spectral densities of large matrices made accurate

Efficient Preconditioning of hp-FEM Matrices by Hierarchical Low-Rank Approximations

Contact Info

Product

Resources

About