Warped Gaussian Processes and Derivative-Based Sequential Designs for Functions with Heterogeneous Variations

Abstract

Gaussian process (GP) models have become popular for approximating and exploring nonlinear systems using scarce input/output samples and prior hypotheses done through mean and covariance functions. While it is common to make stationarity assumptions and use variance-based criteria for exploration, in realistic cases it is not rare that systems under study exhibit a heterogeneous behavior depending on regions of the parameter space. We consider a class of problems where high variations occur along unknown noncanonical directions and we tackle the problem of accommodating nonstationarity from two angles. First we define a novel class of covariances (WaMI-GP) that simultaneously generalizes kernels of multiple index and of tensorized warped GPs, and second, we introduce derivative-based sampling criteria dedicated to the exploration of high-variation regions. The novel GP class is investigated through both mathematical analysis and numerical experiments, and it is shown that it allows encoding much expressiveness while keeping the number of parameters to be inferred moderate. Criteria and models are compared on a mechanical test case from safety studies conducted by IRSN. On this application some of the proposed criteria outperform usual variance-based criteria in the case of a stationary GP model; however, variance-based criteria with WaMI-GP perform even better. Our method is also compared with the treed Gaussian processes (TGP) on this application and on a NASA test case. In the IRSN application, WaMI-GP dominates TGP in static and sequential settings. In the NASA application, while TGP clearly dominates in the static case, for small designs it is outperformed by WaMI-GP in the sequential setup.

Gradient‐based criteria for sequential experiment design

“…Ginsbourger et al 32 use sequential sampling to estimate profile optima. Marmin et al 33 establish a sampling criterion for sequential design based on the variance of the gradient of the Gaussian process model.…”

Active emulation of computer codes with Gaussian processes – Application to remote sensing

“…A commonly used distribution is the truncated Gaussian N T (x|µ, σ, min, max). Indeed, the truncated Gaussians N T (Chl|45, 30,20,90) and N T (LAI|3.5, 4.5, 0, 10) for Chl and LAI, respectively, have proven effective for crop reflectance modeling [7]. We denote their joint distribution, which has no covariance between the variables, as N T (Chl,LAI).…”

“…The reasoning is that areas of high variability in f (x) requires the addition of more information, as has also been noted in [6]. In [20] the predictive variance of the gradient norm of a GP is used as a sampling criteria, which is a less straightforward approach than just using the gradient directly as done here. Similarly, regions with a small concentration of nodes requires the introduction of new nodes in order to fill the space (simple exploration, space filling without taking into account the geometrical features of f (x)).…”

Efficient Global Optimization Using Deep Gaussian Processes

“…However, the deformation in this approach is done only along canonical axes. Marmin et al [34] adress this issue by introducing a parametrized matrix A allowing a linear mapping of the input space before undergoing the non-linear mapping of w(·). The non-linear mapping approach was studied in the context of BO in [35] where it was compared to regular GP.…”