Bayesian Optimal Design of Experiments for Inferring the Statistical Expectation of Expensive Black-Box Functions

Abstract

Bayesian optimal design of experiments (BODE) has been successful in acquiring information about a quantity of interest (QoI) which depends on a black-box function. BODE is characterized by sequentially querying the function at specific designs selected by an infill-sampling criterion. However, most current BODE methods operate in specific contexts like optimization, or learning a universal representation of the black-box function. The objective of this paper is to design a BODE for estimating the statistical expectation of a physical response surface. This QoI is omnipresent in uncertainty propagation and design under uncertainty problems. Our hypothesis is that an optimal BODE should be maximizing the expected information gain in the QoI. We represent the information gain from a hypothetical experiment as the Kullback-Liebler (KL) divergence between the prior and the posterior probability distributions of the QoI. The prior distribution of the QoI is conditioned on the observed data and the posterior distribution of the QoI is conditioned on the observed data and a hypothetical experiment. The main contribution of this paper is the derivation of a semi-analytic mathematical formula for the expected information gain about the statistical expectation of a physical response. The developed BODE is validated on synthetic functions with varying number of input-dimensions. We demonstrate the performance of the methodology on a steel wire manufacturing problem.

Discussion: “Bayesian Optimal Design of Experiments for Inferring the Statistical Expectation of Expensive Black-Box Functions” (Pandita, P., Bilionis, I., and Panchal, J., 2019, ASME J. Mech. Des., 141(10), p. 101404)

“…By using the Github code from the authors of Ref. [1] for their test cases, we have confirmed that the new results based on Eq. ( 15) are the same as the original results based on Eq.…”

“…with s and Λ involving hyperparameters of the kernel function (with either optimized values from training or selected values as in Ref. [1]). The main computation is then Eq.…”

“…In Ref. [1], the authors developed a sequential Bayesian optimal design framework to estimate the statistical expectation of a blackbox function f (x) : R d R. Let x ∼ p(x) with p(x) the probability distribution of the input x, the statistical expectation is then defined as follows:…”

Output-weighted optimal sampling for Bayesian regression and rare event statistics using few samples

“…This allowed us to utilize gradient based optimization methods. For the adaptive strategies based on the mutual information, I(x, y|D, h) and its second-order approximation, I G (x, y|D, h), we used a random sampling approach and equations (12) and (14), respectively. Specifically, we generated 10 4 samples from the input distribution x and utilized the exact expression:…”

“…The problem has been studied extensively using criteria relying on mutual information theory or the Kullback-Leibler (KL) divergence (e.g. [15]). More recently another criterion was introduced focusing on the rapid convergence of the output statistics [16].…”