On Randomized Trace Estimates for Indefinite Matrices with an Application to Determinants

Abstract

Randomized trace estimation is a popular and well-studied technique that approximates the trace of a large-scale matrix B by computing the average of $$x^T Bx$$ x T B x for many samples of a random vector X. Often, B is symmetric positive definite (SPD) but a number of applications give rise to indefinite B. Most notably, this is the case for log-determinant estimation, a task that features prominently in statistical learning, for instance in maximum likelihood estimation for Gaussian process regression. The analysis of randomized trace estimates, including tail bounds, has mostly focused on the SPD case. In this work, we derive new tail bounds for randomized trace estimates applied to indefinite B with Rademacher or Gaussian random vectors. These bounds significantly improve existing results for indefinite B, reducing the number of required samples by a factor n or even more, where n is the size of B. Even for an SPD matrix, our work improves an existing result by Roosta-Khorasani and Ascher (Found Comput Math, 15(5):1187–1212, 2015) for Rademacher vectors. This work also analyzes the combination of randomized trace estimates with the Lanczos method for approximating the trace of f(B). Particular attention is paid to the matrix logarithm, which is needed for log-determinant estimation. We improve and extend an existing result, to not only cover Rademacher but also Gaussian random vectors.

Monte Carlo Methods for Estimating the Diagonal of a Real Symmetric Matrix

“…We present a componentwise absolute error bound (Theorem 5.5) for Gaussian Monte Carlo estimators, and a bound on the minimal sampling amount that makes the Gaussian Monte Carlo estimator a componentwise (ǫ, δ) estimators (Corollary 5.6). Our bounds are derived from and identical to bounds for trace estimators in [7].…”

“…Therefore, estimators for the diagonal of a matrix can be easily adapted to trace estimators. Monte Carlo methods were first proposed by Hutchinson [8], and subsequently improved and expanded to different distributions [2,7,16]. Applications of trace estimators, reviewed in [19], include estimating density of states, log determinants, and Schatten p-norms.…”

The Short-term Rational Lanczos Method and Applications

“…For the polynomial Lanczos method and f (λ) = λ −1 , this was analyzed in [4]; see also [46]. The effectiveness of the overall approach for general functions of symmetric matrices has been established in [61]; see also [47] for an improved method for stochastic trace approximation, and [15] and its references for general randomized approaches.…”

A FEAST SVDsolver based on Chebyshev--Jackson series for computing partial singular triplets of large matrices

“…Cortinovis and Kressner [3] have derived randomized trace estimates for a symmetric indefinite matrix with Rademacher or Gaussian random vectors. Contrary to the SPSD case, their bound is for absolute rather than relative errors of the trace, and the number of samples can be much larger than that in the SPSD case; see the elaborations after Lemma 2.1 in this paper and Remark 4, Corollary 2 of [3] for Rademacher random vectors. Therefore, a reliable and efficient use of the bound in the indefinite case is far from that for the SPSD case.…”

“…the Hutchinson's paper [10]), several algorithms have been proposed to reliably estimate the trace of a matrix by Monte Carlo methods [2,11,32]. As it turns out, the positive semi-definiteness is important and attractive since it enables us to the Rademacher random estimations in [1,3,25] to reliably predict the number n sv of desired singular triplets and propose a practical selection strategy for the subspace dimension in the FEAST SVDsolver.…”