Blendenpik: Supercharging LAPACK's Least-Squares Solver

Avron, Haim; Maymounkov, Petar; Toledo, Sivan

doi:10.1137/090767911

Cited by 171 publications

(269 citation statements)

References 23 publications

Supporting

Mentioning

266

Contrasting

Order By: Relevance

“…Then, in 2010, Avron, Maymounkov, and Toledo [2] implemented a high-precision LS solver, called Blendenpik, and compared it to LAPACK’s DGELS and to LSQR without preconditioning. Blendenpik uses a Walsh–Hadamard transform, a discrete cosine transform, or a discrete Hartley transform for blending the rows/columns, followed by a random sampling, to generate a problem of smaller size.…”

Section: Least Squares Solversmentioning

confidence: 99%

“…It is not necessary for G to be a random normal projection matrix. The result also applies to other random projection matrices satisfying this condition, e.g., the randomized discrete cosine transform used in Blendenpik [2]. …”

Section: Algorithmmentioning

confidence: 99%

“…We implemented our LS solver and compared it with competing solvers: DGELSD/DGELSY from LAPACK [1], spqr_solve (SPQR) from SuiteSparseQR [5, 6], and Blendenpik [2]. Table 1 summarizes the properties of those solvers.…”

Section: Numerical Experimentsmentioning

confidence: 99%

“…In recent years, however, motivated by large data problems, very nontrivial randomized algorithms for very large matrix problems have drawn considerable attention from researchers, originally in theoretical computer science and subsequently in numerical linear algebra and scientific computing. By randomized algorithms, we refer, in particular, to random sampling and random projection algorithms [8, 23, 9, 22, 2]. For a comprehensive overview of these developments, see the review of Mahoney [18], and for an excellent overview of numerical aspects of coupling randomization with classical low-rank matrix factorization methods, see the review of Halko, Martinsson, and Tropp [14].…”

Section: Introductionmentioning

confidence: 99%

“…On large dense systems, is competitive with LAPACK’s DGELSD for strongly overdetermined problems, and it is much faster for strongly underdetermined problems, although solvers using fast random projections, like Blendenpik [2], are still slightly faster in both cases. On sparse systems without sparsity patterns that can be exploited to reduce fill-in (such as matrices with random structure), runs significantly faster than competing solvers, for both the strongly over- or underdetermined cases.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

LSRN: A Parallel Iterative Solver for Strongly Over- or Underdetermined Systems

Meng

Saunders

Mahoney

2014

SIAM J. Sci. Comput.

108

127

View full text Add to dashboard Cite

We describe a parallel iterative least squares solver named that is based on random normal projection. computes the min-length solution to minx∈ℝn ‖Ax − b‖2, where A ∈ ℝm × n with m ≫ n or m ≪ n, and where A may be rank-deficient. Tikhonov regularization may also be included. Since A is involved only in matrix-matrix and matrix-vector multiplications, it can be a dense or sparse matrix or a linear operator, and automatically speeds up when A is sparse or a fast linear operator. The preconditioning phase consists of a random normal projection, which is embarrassingly parallel, and a singular value decomposition of size ⌈γ min(m, n)⌉ × min(m, n), where γ is moderately larger than 1, e.g., γ = 2. We prove that the preconditioned system is well-conditioned, with a strong concentration result on the extreme singular values, and hence that the number of iterations is fully predictable when we apply LSQR or the Chebyshev semi-iterative method. As we demonstrate, the Chebyshev method is particularly efficient for solving large problems on clusters with high communication cost. Numerical results show that on a shared-memory machine, is very competitive with LAPACK’s DGELSD and a fast randomized least squares solver called Blendenpik on large dense problems, and it outperforms the least squares solver from SuiteSparseQR on sparse problems without sparsity patterns that can be exploited to reduce fill-in. Further experiments show that scales well on an Amazon Elastic Compute Cloud cluster.

show abstract

Section: Least Squares Solversmentioning

confidence: 99%

Section: Algorithmmentioning

confidence: 99%

Section: Numerical Experimentsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

LSRN: A Parallel Iterative Solver for Strongly Over- or Underdetermined Systems

Meng

Saunders

Mahoney

2014

SIAM J. Sci. Comput.

108

127

View full text Add to dashboard Cite

show abstract

Large‐scale inverse model analyses employing fast randomized data reduction

Lin

O’Malley

et al. 2017

Water Resources Research

View full text Add to dashboard Cite

When the number of observations is large, it is computationally challenging to apply classical inverse modeling techniques. We have developed a new computationally efficient technique for solving inverse problems with a large number of observations (e.g., on the order of 107 or greater). Our method, which we call the randomized geostatistical approach (RGA), is built upon the principal component geostatistical approach (PCGA). We employ a data reduction technique combined with the PCGA to improve the computational efficiency and reduce the memory usage. Specifically, we employ a randomized numerical linear algebra technique based on a so‐called “sketching” matrix to effectively reduce the dimension of the observations without losing the information content needed for the inverse analysis. In this way, the computational and memory costs for RGA scale with the information content rather than the size of the calibration data. Our algorithm is coded in Julia and implemented in the MADS open‐source high‐performance computational framework (http://mads.lanl.gov). We apply our new inverse modeling method to invert for a synthetic transmissivity field. Compared to a standard geostatistical approach (GA), our method is more efficient when the number of observations is large. Most importantly, our method is capable of solving larger inverse problems than the standard GA and PCGA approaches. Therefore, our new model inversion method is a powerful tool for solving large‐scale inverse problems. The method can be applied in any field and is not limited to hydrogeological applications such as the characterization of aquifer heterogeneity.

show abstract

Preconditioners for Krylov subspace methods: An overview

Pearson

Pestana

2020

GAMM-Mitteilungen

View full text Add to dashboard Cite

When simulating a mechanism from science or engineering, or an industrial process, one is frequently required to construct a mathematical model, and then resolve this model numerically. If accurate numerical solutions are necessary or desirable, this can involve solving large-scale systems of equations. One major class of solution methods is that of preconditioned iterative methods, involving preconditioners which are computationally cheap to apply while also capturing information contained in the linear system. In this article, we give a short survey of the field of preconditioning. We introduce a range of preconditioners for partial differential equations, followed by optimization problems, before discussing preconditioners constructed with less standard objectives in mind.

show abstract

Blendenpik: Supercharging LAPACK's Least-Squares Solver

Cited by 171 publications

References 23 publications

LSRN: A Parallel Iterative Solver for Strongly Over- or Underdetermined Systems

LSRN: A Parallel Iterative Solver for Strongly Over- or Underdetermined Systems

Large‐scale inverse model analyses employing fast randomized data reduction

Preconditioners for Krylov subspace methods: An overview

Contact Info

Product

Resources

About