Bayesian Quadrature, Energy Minimization, and Space-Filling Design

Abstract

A standard objective in computer experiments is to approximate the behavior of an unknown function on a compact domain from a few evaluations inside the domain. When little is known about the function, space-lling design is advisable: typically, points of evaluation spread out across the available space are obtained by minimizing a geometrical (for instance, covering radius) or a discrepancy criterion measuring distance to uniformity. The paper investigates connections between design for integration (quadrature design), construction of the (continuous) BLUE for the location model, space-lling design, and minimization of energy (kernel discrepancy) for signed measures. Integrally strictly positive denite kernels dene strictly convex energy functionals, with an equivalence between the notions of potential and directional derivative, showing the strong relation between discrepancy minimization and more traditional design of optimal experiments. In particular, kernel herding algorithms, which are special instances of vertex-direction methods used in optimal design, can be applied to the construction of point sequences with suitable space-lling properties.

Validation design I: construction of validation designs via kernel herding

“…with respect to measures ζ k of total mass one supported on SS (k) , and one can show (Pronzato and Zhigljavsky, 2020) that its potential k) , 9). At iteration k, the computation of ŵ(k) by ( 11) also induces an additional computational cost of…”

“…denote the energy of ξ for C, so that γ 2 Pronzato and Zhigljavsky (2020), and we can minimize the squared MMD criterion γ 2 C (ξ, µ) = E C (ξ − µ) with respect to ξ by a simple descent algorithm. Denote by F C,µ (ξ; ν) the directional derivative of γ 2 C (•, µ) at ξ in the direction ν,…”

“…One may refer to Szabó and Sriperumbudur (2018) for connections between positive-definiteness properties of the K i and those of K. The expressions of E K i (µ 1 ) and P K i ,µ 1 (•) are available for many kernels K i ; see Pronzato and Zhigljavsky (2020) and the references therein.…”

“…However, it does not provide the optimal design for m fixed: the construction of one-shot m-point designs minimizing a MMD criterion is considered for instance in(Pronzato and Zhigljavsky, 2020); we do not develop this aspect here.…”

Incremental Space-Filling Design Based on Coverings and Spacings: Improving Upon Low Discrepancy Sequences

“…1 There exist other incremental constructions that generate space-filling designs with good covering performance, in particular those based on maximization of mutual information [2,17] or on minimization of a kernel discrepancy by kernel herding [33]. They rely on the choice of a suitable positive definite kernel and are especially adapted to interpolation based on Gaussian process models.…”