Exploiting Low Rank Covariance Structures for Computing High-Dimensional Normal and Student-$t$ Probabilities

Cao, Jian; Genton, Marc G.; Keyes, David E.; Turkiyyah, George

doi:10.48550/arxiv.2003.11183

Cited by 1 publication

(10 citation statements)

References 35 publications

(58 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Evaluation of (4) requires the calculation of cumulative distribution functions of multivariate Gaussians, which is known to be a challenging task in high dimensions (e.g., Genz, 1992;Chopin, 2011;Botev, 2017;Genton et al, 2018;Cao et al, 2019Cao et al, , 2020. Recent advances via minimax tilting methods (Botev, 2017) allow accurate evaluation of such quantities, but face an increased computational cost which makes such strategies rapidly impractical as n grows.…”

Section: Evaluation Via Multivariate Gaussian Probability Ratiosmentioning

confidence: 99%

“…Recent advances via minimax tilting methods (Botev, 2017) allow accurate evaluation of such quantities, but face an increased computational cost which makes such strategies rapidly impractical as n grows. A more scalable solution can be found in the separation-of-variable (sov) algorithm originally introduced by Genz (1992) and subsequently improved in terms of scalability by Cao et al (2020). Such a routine decomposes the generic multivariate…”

Section: Evaluation Via Multivariate Gaussian Probability Ratiosmentioning

confidence: 99%

“…This ordering strategy processes the integration variables from left to right iteratively and, at each step, it switches the original integration variable with the one having the narrowest conditional integration limits on its right side. Positioning left the integration variables with narrower integration limits is shown in Trinh and Genz (2015) and Cao et al (2020) to improve the Monte Carlo convergence rate of (5), whose integrand is evaluated R times -corresponding to the Monte Carlo sample size -each of which has a cost of O(n 2 ). Such costs allow the implementation of this strategy in settings with n ≤ 1,000, thus motivating more scalable options in high dimensions.…”

Section: Evaluation Via Multivariate Gaussian Probability Ratiosmentioning

confidence: 99%

“…Such costs allow the implementation of this strategy in settings with n ≤ 1,000, thus motivating more scalable options in high dimensions. Cao et al (2020) address this issue via a tile-low-rank representation for Σ that reduces the cost of the sov algorithm by substituting the dense matrix-vector multiplication with the low-rank matrix-vector multiplication. A compatible block-reordering is also introduced in place of the univariate reordering to improve the convergence rate at the same cost as the low-rank Cholesky factorization.…”

Section: Evaluation Via Multivariate Gaussian Probability Ratiosmentioning

confidence: 99%

“…More specifically, in Section 2.1 we first derive a closed-form expression for the marginal likelihood p(y) under model (1), and then exploit this result to show that p(y n+1 = 1 | y) can be expressed as the ratio between cumulative distribution functions of multivariate Gaussians with dimensions n + 1 and n, respectively. To overcome the known issues associated with the evaluation of these two quantities in high dimensions (Chopin, 2011;Botev, 2017;Cao et al, 2019Cao et al, , 2020 we introduce an error-reduction technique for computing ratios of Gaussian cumulative distribution functions that builds on the tile-low-rank method in Cao et al (2020) and substantially reduces the computational time of state-of-the-art strategies such as minimax tilting methods (Botev, 2017) and Hamiltonian Monte Carlo samplers (stan) (Hoffman and Gelman, 2014), without affecting accuracy.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Scalable computation of predictive probabilities in probit models with Gaussian process priors

Cao¹,

Durante²,

Genton³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Predictive models for binary data are fundamental in various fields, and the growing complexity of modern applications has motivated several flexible specifications for modeling the relationship between the observed predictors and the binary responses. A widely-implemented solution expresses the probability parameter via a probit mapping of a Gaussian process indexed by predictors. However, unlike for continuous settings, there is a lack of closed-form results for predictive distributions in binary models with Gaussian process priors. Markov chain Monte Carlo methods and approximation strategies provide common solutions to this problem, but state-of-the-art algorithms are either computationally intractable or inaccurate in moderate-to-high dimensions. In this article, we aim to cover this gap by deriving closed-form expressions for the predictive probabilities in probit Gaussian processes that rely either on cumulative distribution functions of multivariate Gaussians or on functionals of multivariate truncated normals. To evaluate such quantities we develop novel scalable solutions based on tile-low-rank Monte Carlo methods for computing multivariate Gaussian probabilities, and on variational approximations of multivariate truncated normals. Closed-form expressions for marginal likelihoods and posterior distributions of the Gaussian process are also discussed. As illustrated in empirical studies, the proposed methods scale to dimensions where state-of-the-art solutions are impractical.

show abstract