Parallel multivariate slice sampling

Tibbits, M.; Haran, Murali; Liechty, John

doi:10.1007/s11222-010-9178-z

Cited by 28 publications

(32 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…More recent work combines the use of multiple chains with adaptive MCMC in an attempt to use these multiple sources of information to learn an appropriate proposal distribution (12,13). Sometimes, specific MCMC algorithms are directly amenable to parallelization, such as independent Metropolis−Hastings (14) or slice sampling (15), as indeed are some statistical models via careful reparameterization (16) or implementation on specialist hardware, such as graphics processing units (GPUs) (17,18); however, these approaches are often problem specific and not generally applicable. For problems involving large amounts of data, parallelization may in some cases also be possible by partitioning the data and analyzing each subset using standard MCMC methods simultaneously on multiple machines (19).…”

mentioning

confidence: 99%

A general construction for parallelizing Metropolis−Hastings algorithms

Calderhead

2014

Proc. Natl. Acad. Sci. U.S.A.

117

130

View full text Add to dashboard Cite

Markov chain Monte Carlo methods (MCMC) are essential tools for solving many modern-day statistical and computational problems; however, a major limitation is the inherently sequential nature of these algorithms. In this paper, we propose a natural generalization of the Metropolis−Hastings algorithm that allows for parallelizing a single chain using existing MCMC methods. We do so by proposing multiple points in parallel, then constructing and sampling from a finite-state Markov chain on the proposed points such that the overall procedure has the correct target density as its stationary distribution. Our approach is generally applicable and straightforward to implement. We demonstrate how this construction may be used to greatly increase the computational speed and statistical efficiency of a variety of existing MCMC methods, including Metropolis-Adjusted Langevin Algorithms and Adaptive MCMC. Furthermore, we show how it allows for a principled way of using every integration step within Hamiltonian Monte Carlo methods; our approach increases robustness to the choice of algorithmic parameters and results in increased accuracy of Monte Carlo estimates with little extra computational cost. S ince its introduction in the 1970s, the Metropolis−Hastings algorithm has revolutionized computational statistics (1). The ability to draw samples from an arbitrary probability distribution, πðXÞ, known only up to a constant, by constructing a Markov chain that converges to the correct stationary distribution has enabled the practical application of Bayesian inference for modeling a huge variety of scientific phenomena, and has resulted in Metropolis−Hastings being noted as one of the top 10 most important algorithms from the 20th century (2). Despite regular increases in available computing power, Markov chain Monte Carlo (MCMC) algorithms can still be computationally very expensive; many thousands of iterations may be necessary to obtain low-variance estimates of the required quantities with an oftentimes complex statistical model being evaluated for each set of proposed parameters. Furthermore, many Metropolis−Hastings algorithms are severely limited by their inherently sequential nature.Many approaches have been proposed for improving the statistical efficiency of MCMC, and although such algorithms are guaranteed to converge asymptotically to the stationary distribution, their performance over a finite number of iterations can vary hugely. Much research effort has therefore focused on developing transition kernels that enable moves to be proposed far from the current point and subsequently accepted with high probability, taking into account, for example, the correlation structure of the parameter space (3, 4), or by using Hamiltonian dynamics (5) or diffusion processes (6). A recent investigation into proposal kernels suggests that more exotic distributions, such as the Bactrian kernel, might also be used to increase the statistical efficiency of MCMC algorithms (7). The efficient exploration of high-dimensional and multimo...

show abstract

mentioning

confidence: 99%

A general construction for parallelizing Metropolis−Hastings algorithms

Calderhead

2014

Proc. Natl. Acad. Sci. U.S.A.

117

130

View full text Add to dashboard Cite

show abstract

“…Nevertheless, parallel computing can be use to accelerate some calculations inside the MCMC steps, for example, those involved in likelihood evaluations (Tibbits et al, 2011 …”

Section: Discussionmentioning

confidence: 99%

Optimal direction Gibbs sampler for truncated multivariate normal distributions

Christen

Fox

Santana-Cibrian

2016

Communications in Statistics - Simulation and Computation

View full text Add to dashboard Cite

Generalized Gibbs samplers simulate from any direction, not necessarily limited to the coordinate directions of the parameters of the objective function. We study how to optimally choose such directions in a random scan Gibbs sampler setting. We consider that optimal directions will be those that minimize the Kullback-Leibler divergence of two Markov chain Monte Carlo steps. Two distributions over direction are proposed for the multivariate Normal objective function. The resulting algorithms are used to simulate from a truncated multivariate Normal distribution, and the performance of our algorithms is compared with the performance of two algorithms based on the Gibbs sampler.

show abstract

“…According to the results, by using this method, one can generate approximately independent and identically distributed samples at a rate that is more efficient than other methods that update all dimensions at once. Additionally, Tibbits et al [40] propose an approach to multivariate slice sampler that naturally lends itself to a parallel implementation. They study approaches for constructing a multivariate slice sampler, and they show how parallel computing can be useful for making MCMC algorithms computationally efficient.…”

Section: Slice Samplermentioning

confidence: 99%

“…They study approaches for constructing a multivariate slice sampler, and they show how parallel computing can be useful for making MCMC algorithms computationally efficient. Tibbits et al [40] examine various implementations of their algorithm in the context of real and simulated data. Moreover, Kalli et al [41] present a more efficient version of the slice sampler for Dirichlet process mixture models described by Walker [37].…”

Section: Slice Samplermentioning

confidence: 99%

Slice sampler algorithm for generalized Pareto distribution

Rostami¹,

Yahya²,

Yahya³

et al. 2017

HJMS

View full text Add to dashboard Cite

In this paper, we developed the slice sampler algorithm for the generalized Pareto distribution (GPD) model. Two simulation studies have shown the performance of the peaks over given threshold (POT) and GPD density function on various simulated data sets. The results were compared with another commonly used Markov chain Monte Carlo (MCMC) technique called Metropolis-Hastings algorithm. Based on the results, the slice sampler algorithm provides closer posterior mean values and shorter 95% quantile based credible intervals compared to the Metropolis-Hastings algorithm. Moreover, the slice sampler algorithm presents a higher level of stationarity in terms of the scale and shape parameters compared with the Metropolis-Hastings algorithm. Finally, the slice sampler algorithm was employed to estimate the return and risk values of investment in Malaysian gold market.

show abstract

Parallel multivariate slice sampling

Cited by 28 publications

References 33 publications

A general construction for parallelizing Metropolis−Hastings algorithms

A general construction for parallelizing Metropolis−Hastings algorithms

Optimal direction Gibbs sampler for truncated multivariate normal distributions

Slice sampler algorithm for generalized Pareto distribution

Contact Info

Product

Resources

About