Fast Bayesian experimental design: Laplace-based importance sampling for the expected information gain

Abstract

In calculating expected information gain in optimal Bayesian experimental design, the computation of the inner loop in the classical double-loop Monte Carlo requires a large number of samples and suffers from underflow if the number of samples is small. These drawbacks can be avoided by using an importance sampling approach. We present a computationally efficient method for optimal Bayesian experimental design that introduces importance sampling based on the Laplace method to the inner loop. We derive the optimal values for the method parameters in which the average computational cost is minimized according to the desired error tolerance. We use three numerical examples to demonstrate the computational efficiency of our method compared with the classical double-loop Monte Carlo, and a more recent single-loop Monte Carlo method that uses the Laplace method as an approximation of the return value of the inner loop. The first example is a scalar problem that is linear in the uncertain parameter. The second example is a nonlinear scalar problem. The third example deals with the optimal sensor placement for an electrical impedance tomography experiment to recover the fiber orientation in laminate composites.

Multilevel Monte Carlo estimation of expected information gains

“…Much more recently, in [1], Beck et al provided a thorough error analysis of the NMC estimator and derived the optimal allocation of N and M for a given ε. In fact, they considered the situation where g cannot be computed exactly and only its discretized approximation g h with a mesh discretization parameter h is available.…”

“…This means that the expected information gain U ξ measures the average amount of the reduction of the information entropy about θ by collecting data Y ξ . In (1), the inner expectation appearing in the right-most side is nothing but the Kullback-Leibler divergence between p(θ) and p(θ | Y ξ ). In the context of Bayesian experimental designs, we claim that the data Y ξ with larger value of U ξ is more informative about θ and thus the corresponding experimental design ξ is better.…”

“…Typically ǫ is assumed to be zero-mean Gaussian with covariance matrix Σ ǫ . As considered in [13,15,1], this data model can be extended to allow the repetition of experiments as…”

Multilevel double loop Monte Carlo and stochastic collocation methods with importance sampling for Bayesian optimal experimental design

“…In Figure 6, we compare the computational time of MLDLSC and the average time of the MLDLMC runs for a range of error tolerances. We also include an estimate of the computational time for DLMCIS method proposed in [6]. MLDLSC performs better than MLDLMC, because the polynomial approximations of MLDLSC takes advantage of the regularity of the expected information gain with respect to the random parameters.…”

“…We present two new computationally efficient methods of approximating the expected information gain based on the Kullback-Leibler divergence, in the context of Bayesian optimal experimental design. The first method we propose is a multilevel double loop Monte Carlo (MLDLMC), which improves upon the double loop Monte Carlo importance sampling (DLMCIS) method in [6].…”