Optimal Query Complexity for Estimating the Trace of a Matrix

Wimmer, Karl; Wu, Yi; Zhang, Peng

doi:10.1007/978-3-662-43948-7_87

Cited by 12 publications

(14 citation statements)

References 12 publications

(14 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• We prove that no practical deterministic algorithm, in a meaningful sense, could possibly be used to estimate the preconditioner stability I −M −1 A F . • We confirm the conjecture of [5] regarding the true asymptotic sample complexity of the Gaussian trace estimator using a substantially more direct proof than the general result provided in [33], at the same time confirming the tightness of our stability estimation algorithm convergence bound. • We provide a randomized algorithm which can provably select a preconditioner of approximately minimal stability among n candidate preconditioners using computational resources equivalent to computing on the order of n log n steps of the conjugate gradients algorithm.…”

Section: Introductionsupporting

confidence: 82%

“…In Section 2.4 we take advantage of highly informative results from the literature on trace estimation to provide useful approximation guarantees and runtime bounds for the previously presented algorithms. Section 2.4.1 returns the focus to Theorem 2.2 of Section 2.4, showing that the leading constant is tight and providing a more concise resolution of a conjecture from [5] than the more generalized result from [33]. In that section, we also comment that no randomized algorithm for estimating preconditioner stability which has access to matrix-vector products of the form (I − M −1 A)z could possibly do better asymptotically than Algorithm 1 by relying on this result [33,Thm.…”

Section: Algorithmsmentioning

confidence: 99%

See 1 more Smart Citation

A Randomized Algorithm for Preconditioner Selection

DiPaolo,

2019

Preprint

View full text Add to dashboard Cite

The task of choosing a preconditioner M to use when solving a linear system Ax = b with iterative methods is difficult. For instance, even if one has access to a collection M 1 , M 2 , . . . , M n of candidate preconditioners, it is currently unclear how to practically choose the M i which minimizes the number of iterations of an iterative algorithm to achieve a suitable approximation to x. This paper makes progress on this sub-problem by showing that the preconditioner stability I − M −1 A F , known to forecast preconditioner quality, can be computed in the time it takes to run a constant number of iterations of conjugate gradients through use of sketching methods. This is in spite of folklore which suggests the quantity is impractical to compute, and a proof we give that ensures the quantity could not possibly be approximated in a useful amount of time by a deterministic algorithm. Using our estimator, we provide a method which can provably select the minimal stability preconditioner among n candidates using floating point operations commensurate with running on the order of n log n steps of the conjugate gradients algorithm. Our method can also advise the practitioner to use no preconditioner at all if none of the candidates appears useful. The algorithm is extremely easy to implement and trivially parallelizable. In one of our experiments, we use our preconditioner selection algorithm to create to the best of our knowledge the first preconditioned method for kernel regression reported to never use more iterations than the non-preconditioned analog in standard tests.

show abstract

Section: Introductionsupporting

confidence: 82%

Section: Algorithmsmentioning

confidence: 99%

A Randomized Algorithm for Preconditioner Selection

DiPaolo,

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…Figure 1 illustrates this growth. In the case of relative error estimates for symmetric positive semidefinite (SPSD) matrices, it is shown in [44] that the dependence on log 2 δ and 1 ε 2 cannot be improved. Remark 2 For a nonzero SPSD matrix B, the result of Theorem 1 can be turned into a relative error estimate.…”

Section: Lemma 4 Let N Be Even and Consider The Traceless Matrix Bmentioning

confidence: 99%

On Randomized Trace Estimates for Indefinite Matrices with an Application to Determinants

Cortinovis

Kreßner

2021

Found Comput Math

View full text Add to dashboard Cite

Randomized trace estimation is a popular and well-studied technique that approximates the trace of a large-scale matrix B by computing the average of $$x^T Bx$$ x T B x for many samples of a random vector X. Often, B is symmetric positive definite (SPD) but a number of applications give rise to indefinite B. Most notably, this is the case for log-determinant estimation, a task that features prominently in statistical learning, for instance in maximum likelihood estimation for Gaussian process regression. The analysis of randomized trace estimates, including tail bounds, has mostly focused on the SPD case. In this work, we derive new tail bounds for randomized trace estimates applied to indefinite B with Rademacher or Gaussian random vectors. These bounds significantly improve existing results for indefinite B, reducing the number of required samples by a factor n or even more, where n is the size of B. Even for an SPD matrix, our work improves an existing result by Roosta-Khorasani and Ascher (Found Comput Math, 15(5):1187–1212, 2015) for Rademacher vectors. This work also analyzes the combination of randomized trace estimates with the Lanczos method for approximating the trace of f(B). Particular attention is paid to the matrix logarithm, which is needed for log-determinant estimation. We improve and extend an existing result, to not only cover Rademacher but also Gaussian random vectors.

show abstract

“…‡ For example, one can use normally distributed variables and define the Gaussian estimator exactly in the same fashion as in (3). While the variance of such an estimator is larger than that of Hutchinson, which uses Rademacher vectors, it shows a better convergence to the trace, in terms of the number of sample vectors n v [22][23][24].…”

Section: Introductionmentioning

confidence: 99%

Efficient estimation of eigenvalue counts in an interval

Napoli

Polizzi

Saad

2016

Numerical Linear Algebra App

110

142

View full text Add to dashboard Cite

Estimating the number of eigenvalues located in a given interval of a large sparse Hermitian matrix is an important problem in certain applications, and it is a prerequisite of eigensolvers based on a divide-andconquer paradigm. Often, an exact count is not necessary, and methods based on stochastic estimates can be utilized to yield rough approximations. This paper examines a number of techniques tailored to this specific task. It reviews standard approaches and explores new ones based on polynomial and rational approximation filtering combined with a stochastic procedure. We also discuss how the latter method is particularly wellsuited for the FEAST eigensolver.ESTIMATING EIGENVALUE COUNTS 675 count OEa; b in OEa; b. While this method yields an exact count, it requires two complete LDL T factorizations, and this can be quite expensive for realistic eigenproblems.This paper discusses two alternative methods that provide only an estimate for OEa; b , but which are relatively inexpensive. Both methods work by estimating the trace of the spectral projector P associated with the eigenvalues inside the interval OEa; b. This spectral projector is expanded in two different ways, and its trace is computed by resorting to stochastic trace estimators, for example, [10,11]. The first method utilizes filtering techniques based on Chebyshev polynomials. The resulting projector is expanded as a polynomial function of A. In the second method, the projector is constructed by integrating the resolvent of the eigenproblem along a contour in the complex plane enclosing the interval OEa; b. In this case, the projector is approximated by a rational function of A.For each of the aforementioned methods, we present various implementations depending on the nature of the eigenproblem (generalized versus standard) and cost considerations. Thus, in the polynomial expansion case, we propose a barrier-type filter when dealing with a standard eigenproblem and two high/low pass filters in the case of generalized eigenproblems. In the rational expansion case, we have the choice of using an LU factorization or a Krylov subspace method to solve linear systems. The optimal implementation of each method used for the eigenvalue count depends on the situation at hand and involves compromises between cost and accuracy. While it is not the aim of this paper to explore detailed analysis of these techniques, we will discuss various possibilities and provide illustrative examples.The polynomial and rational expansion methods are motivated by two distinct approaches recently suggested in the context of electronic structure calculations: (i) spectrum slicing and (ii) Cauchy integral eigen-projection. In the spectrum slicing techniques [1], the eigenpairs are computed by dividing the spectrum in many small subintervals, called 'slices' or 'windows'. For each window, a barrier function is approximated by Chebyshev-Jackson polynomials in order to select only the portion of the spectrum in the slice. In this method, it is important to determine an approximate c...

show abstract

Optimal Query Complexity for Estimating the Trace of a Matrix

Cited by 12 publications

References 12 publications

A Randomized Algorithm for Preconditioner Selection

A Randomized Algorithm for Preconditioner Selection

On Randomized Trace Estimates for Indefinite Matrices with an Application to Determinants

Efficient estimation of eigenvalue counts in an interval

Contact Info

Product

Resources

About