Bayesian model comparison with un-normalised likelihoods

Abstract

Models for which the likelihood function can be evaluated only up to a parameter-dependent unknown normalising constant, such as Markov random field models, are used widely in computer science, statistical physics, spatial statistics, and network analysis. However, Bayesian analysis of these models using standard Monte Carlo methods is not possible due to the intractability of their likelihood functions. Several methods that permit exact, or close to exact, simulation from the posterior distribution have recently been developed. However, estimating the evidence and Bayes' factors (BFs) for these models remains challenging in general. This paper describes new random weight importance sampling and sequential Monte Carlo methods for estimating BFs that use simulation to circumvent the evaluation of the intractable likelihood, and compares them to existing methods. In some cases we observe an advantage in the use of biased weight estimates. An initial investigation into the theoretical and empirical properties of this class of methods is presented. Some support for the use of biased estimates is presented, but we advocate caution in the use of such estimates

Perturbation bounds for Monte Carlo within Metropolis via restricted approximations

“…Their algorithm is motivated by the exchange algorithm [MGM06] which, perhaps surprisingly, can avoid the need for evaluating the ratio Z(x)/Z(y) and targets the distribution π exactly, see e.g. [EJREH17,PH18] for an overview of these and related methods. However, in some cases the exchange algorithm performs poorly, see [AFEB16].…”

Bayesian Model Selection for Exponential Random Graph Models via Adjusted Pseudolikelihoods

“…Recent studies on the inference of ERGMs with the Bayesian approach include Koskinen (2004), Caimo and Friel (2011), Wang and Atchade (2014), Caimo and Mira (2015), Thiemichen et al (2016) and Bouranis et al (2017). Bayesian model selection for exponential random graph models has been explored by Caimo and Friel (2013), Friel (2013), Thiemichen et al (2016) and Everitt et al (2017).…”

“…Related work by Friel (2013) and Everitt et al (2017) has the same objective as our study, namely to estimate the evidence in the presence of an intractable likelihood normalising constant.…”

Efficient importance sampling in low dimensions using affine arithmetic

“…-In some situations a target distribution may not be available to evaluate pointwise directly. For example, consider the situation where the target distribution is the marginal posterior distribution of a parameter (integrating over a large number of latent variables) (Tran et al 2014), or in the situation of a likelihood with an intractable partition function (Everitt et al 2017). Along the same lines as the pseudo-marginal approach, we may use an importance sampler within an importance sampler, with the internal importance sampler serving to estimate the unavailable intractable likelihood.…”