Direct Sampling

Abstract

In recent years, Markov chain Monte Carlo (MCMC) methods have been used to provide a full Bayesian analysis both when the posterior distribution of interest is analytically intractable, and it is not known how to draw independent samples. In this article, a non-MCMC approach to sampling from posterior distributions is developed and illustrated. Some sampling problems, now thought to be best handled by MCMC methods alone, are tackled efficiently via independent samples. This article has supplementary material online.

Scalable Rejection Sampling for Bayesian Hierarchical Models

“…Like our method, Direct Sampling removes the need to concern oneself with issues like chain convergence and autocorrelation, and generates independent samples from a target posterior distribution in parallel. Walker et al (2011) also prove that the sample acceptance probabilities using Direct Sampling are better than those from standard rejection algorithms. Put simply, for many common Bayesian models, they demonstrate improvement over MCMC in terms of efficiency, resource demands and ease of implementation.…”

“…In Equation 8, we see that p(u|y) is proportional to the function q(u). Walker et al (2011) sample from a similar kind of density by first taking M proposal draws from the prior to construct an empirical approximation to q(u), and then approximating that continuous density using Bernstein polynomials. However, in high-dimensional models, this approximation tends to be a poor one at the endpoints, even with an extremely large number of Bernstein polynomial components.…”

“…In contrast, we accept a discrete approximation of p(u|y) in exchange for higher acceptance probabilities. Walker et al (2011) introduced and demonstrated the merits of a non-MCMC approach called Direct Sampling for conducting Bayesian inference. Like our method, Direct Sampling removes the need to concern oneself with issues like chain convergence and autocorrelation, and generates independent samples from a target posterior distribution in parallel.…”

Direct Sampling of Bayesian Thin-Plate Splines for Spatial Smoothing

“…Inference for Bayesian models is based on a using the posterior distribution which can typically not be obtained analytically and requires using methods to draw samples from the posterior distribution. Ideally these samples are drawn directly from the joint posterior of all the model parameters, but in practice this is typically not possible, necessitating the use of Monte Carlo Markov Chain (MCMC) schemes which produce correlated samples reducing effective sample size, and can require a substantial "burn-in" to ensure that the samples are drawn from the target distribution (Walker et al, 2011). In some cases, Bayesian models, including spatial smoothers can be evaluated using integrated-nested Laplace approximations (INLA) methods (Rue et al, 2009), but these methods rely on approximations to the posterior distributions.…”

Generalized Direct Sampling for Hierarchical Bayesian Models

“…Recently, Walker et. al (2010) introduced and demonstrated the merits of a non-MCMC approach called Direct Sampling (DS) for conducting Bayesian inference.…”