Simulating normalizing constants: from importance sampling to bridge sampling to path sampling

Estimating Bayes factors via thermodynamic integration and population MCMC

“…Path sampling methods (Gelman and Meng, 1998) have been shown to perform well at the task of estimating marginal likelihoods (Friel and Pettitt, 2008;Lartillot and Philippe, 2006). Such methods rely on sampling from a sequence of distributions which form a "bridge" in the probability density space connecting the prior distribution to the posterior distribution, and integrating over them.…”

“…It is widely accepted that estimating the marginal likelihood of a non-trivial statistical model is generally very challenging and methods employing some form of thermodynamic integration or path sampling (Gelman and Meng, 1998), although computationally more expensive than the importance sampling methods previously described, have been shown to perform well on a number of forms of statistical model, see for example (Friel and Pettitt, 2008), or (Lartillot and Philippe, 2006) for use in a phylogenetic context.…”

“…Additionally, we can derive an expression for the optimal temperature schedule in terms of minimising the Monte Carlo variance (Gelman and Meng, 1998). We firstly define a density p(t) over the temperature values…”

“…We note that although other sequences are also possible (Gelman and Meng, 1998), this particular formulation allows us to sample from all the required power posteriors simultaneously, which may be later employed in thermodynamic integration. A time homogeneous Markov transition kernel which has p(θ ) as its stationary distribution can be constructed from both local proposal moves and global moves between the tempered chains of the population (Liang and Wong, 2001;Laskey and Myers, 2003;Jasra et al, 2007) thus allowing freer movement within the parameter space.…”

Bayes factors for detection of Quantitative Trait Loci

“…Unfortunately, the exact calculation of general BF is not feasible for relatively complex models [19]. For this reason, Monte Carlo methods, such as the Harmonic Mean Estimation [24] or the Monte Carlo marginal likelihood [3], have been developed, as reviewed by Gelman and Meng [7] and Han and Carlin [11]. Moreover, some other alternatives for providing posterior probabilities have been suggested [4,8].…”