Fast computation of high-dimensional multivariate normal probabilities

Phinikettos, Ioannis

doi:10.1016/j.csda.2010.10.005

Cited by 15 publications

(7 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Quasi-randomized Monte Carlo methods (Genz, 1992;Genz & Bretz, 2009;Botev, 2017) have been proposed to calculate s+m a for small m a (few hundreds observations). These approaches are not in general suitable for medium and large m a (apart from special cases Phinikettos & Gandy, 2011;Genton et al, 2018;Azzimonti & Ginsbourger, 2018). Another approach is to use EP approximations as in Cunningham et al (2013), which has a similar computational load of GP-EP for learning the hyperparameters.…”

Section: Sampling From the Posterior And Hyperparameters Selectionmentioning

confidence: 99%

A unified framework for closed-form nonparametric regression, classification, preference and mixed problems with Skew Gaussian Processes

2021

View full text Add to dashboard Cite

Skew-Gaussian Processes (SkewGPs) extend the multivariate Unified Skew-Normal distributions over finite dimensional vectors to distribution over functions. SkewGPs are more general and flexible than Gaussian processes, as SkewGPs may also represent asymmetric distributions. In a recent contribution, we showed that SkewGP and probit likelihood are conjugate, which allows us to compute the exact posterior for non-parametric binary classification and preference learning. In this paper, we generalize previous results and we prove that SkewGP is conjugate with both the normal and affine probit likelihood, and more in general, with their product. This allows us to (i) handle classification, preference, numeric and ordinal regression, and mixed problems in a unified framework; (ii) derive closed-form expression for the corresponding posterior distributions. We show empirically that the proposed framework based on SkewGP provides better performance than Gaussian processes in active learning and Bayesian (constrained) optimization. These two tasks are fundamental for design of experiments and in Data Science.

show abstract

Section: Sampling From the Posterior And Hyperparameters Selectionmentioning

confidence: 99%

A unified framework for closed-form nonparametric regression, classification, preference and mixed problems with Skew Gaussian Processes

2021

View full text Add to dashboard Cite

show abstract

“…Quasi-randomized Monte Carlo methods (Genz, 1992;Genz and Bretz, 2009;Botev, 2017) have been proposed to calculate Φ s+m a for small m a (few hundreds observations). These approaches are not in general suitable for medium and large m a (apart from special cases Phinikettos and Gandy (2011);Genton et al (2018); Azzimonti and Ginsbourger (2018)). We overcome this issue by using the approximation introduced in (Benavoli et al, 2020a, Prop.2):…”

Section: Computing the Marginal Likelihood And Sampling From The Post...mentioning

confidence: 99%

A unified framework for closed-form nonparametric regression, classification, preference and mixed problems with Skew Gaussian Processes

Benavoli¹,

Azzimonti²,

Piga³

2020

Preprint

View full text Add to dashboard Cite

Skew-Gaussian processes (SkewGPs) extend the multivariate Unified Skew-Normal distributions over finite dimensional vectors to distribution over functions. SkewGPs are more general and flexible than Gaussian processes, as SkewGPs may also represent asymmetric distributions. In a recent contribution we showed that SkewGP and probit likelihood are conjugate, which allows us to compute the exact posterior for non-parametric binary classification and preference learning. In this paper, we generalize previous results and we prove that SkewGP is conjugate with both the normal and affine probit likelihood, and more in general, with their product. This allows us to (i) handle classification, preference, numeric and ordinal regression, and mixed problems in a unified framework; (ii) derive closed-form expression for the corresponding posterior distributions. We show empirically that the proposed framework based on SkewGP provides better performance than Gaussian processes in active learning and Bayesian (constrained) optimization.

show abstract

“…These methods are not adequate for high-dimensional problems, however, as they scale poorly with dimension. Phinikettos and Gandy (2011) showed the effectiveness of a number of variance Botev (2016) proposed the use of an exponential tilting importance sampling strategy in Monte Carlo integration and showed its remarkable effectiveness in decreasing the variance of the estimates, particularly in the tails of the distribution compared to Genz' strategy. Conditioning approximations (Mendell and Elston, 1974;Trinh and Genz, 2015) can produce fast estimates of multivariate normal probabilities but unfortunately cannot produce estimates of the approximation errors.…”

Section: Accepted Manuscriptmentioning

confidence: 99%

Hierarchical Decompositions for the Computation of High-Dimensional Multivariate Normal Probabilities

Genton

Keyes

Turkiyyah

2018

Journal of Computational and Graphical Statistics

View full text Add to dashboard Cite

We present a hierarchical decomposition scheme for computing the n-dimensional integral of multivariate normal probabilities that appear frequently in statistics. The scheme exploits the fact that the formally dense covariance matrix can be approximated by a matrix with a hierarchical low rank structure. It allows the reduction of the computational complexity per Monte Carlo sample from O(n 2) to O(mn + knlog(n/m)), where k is the numerical rank of off-diagonal matrix blocks and m is the size of small diagonal blocks in the matrix that are not well-approximated by low rank factorizations and treated as dense submatrices. This hierarchical decomposition leads to substantial efficiencies in multivariate normal probability computations and allows integrations in thousands of dimensions to be practical on modern workstations.

show abstract

Fast computation of high-dimensional multivariate normal probabilities

Cited by 15 publications

References 14 publications

A unified framework for closed-form nonparametric regression, classification, preference and mixed problems with Skew Gaussian Processes

A unified framework for closed-form nonparametric regression, classification, preference and mixed problems with Skew Gaussian Processes

A unified framework for closed-form nonparametric regression, classification, preference and mixed problems with Skew Gaussian Processes

Hierarchical Decompositions for the Computation of High-Dimensional Multivariate Normal Probabilities

Contact Info

Product

Resources

About