Revisiting the balance heuristic for estimating normalising constants

Abstract

Multiple importance sampling estimators are widely used for computing intractable constants due to its reliability and robustness. The celebrated balance heuristic estimator belongs to this class of methods and has proved very successful in computer graphics. The basic ingredients for computing the estimator are: a set of proposal distributions, indexed by some discrete label, and a predetermined number of draws from each of these proposals. However, if the number of available proposals is much larger than the number of permitted importance points, one needs to select, possibly at random, which of these distributions will be used. The focus of this work lies within the previous context, exploring some improvements and variations of the balance heuristic via a novel extended-space representation of the estimator, leading to straightforward annealing schemes for variance reduction purposes. In addition, we also look at the intractable scenario where the proposal density is only available as a joint function with the discrete label, as may be encountered in problems where an ordering is imposed. For this case, we look at combinations of correlated unbiased estimators which also fit into the extended-space representation and, in turn, will provide other interesting solutions.

Advances in Importance Sampling

“…When the selection of the proposals is also random, unlike in the sampling in (17), there exist options to evaluate only the proposals that have been used for sampling (scheme R2 in [42]) instead of using all of them in the numerator (scheme R3 in [42]). A recent paper explores the R2 scheme and some of its statistical properties [87].…”

Advances in Importance Sampling

“…When the selection of the proposals is also random, unlike in the sampling in (17), there exist options to evaluate only the proposals that have been used for sampling (scheme R2 in [40]) instead of using all of them in the numerator (scheme R3 in [40]). A recent paper explores the R2 scheme and some of its statistical properties [77].…”