Vecchia-approximated Deep Gaussian Processes for Computer Experiments

Abstract

Deep Gaussian processes (DGPs) upgrade ordinary GPs through functional composition, in which intermediate GP layers warp the original inputs, providing flexibility to model non-stationary dynamics. Two DGP regimes have emerged in recent literature. A "big data" regime, prevalent in machine learning, favors approximate, optimization-based inference for fast, high-fidelity prediction. A "small data" regime, preferred for computer surrogate modeling, deploys posterior integration for enhanced uncertainty quantification (UQ). We aim to bridge this gap by expanding the capabilities of Bayesian DGP posterior inference through the incorporation of the Vecchia approximation, allowing linear computational scaling without compromising accuracy or UQ. We are motivated by surrogate modeling of simulation campaigns with upwards of 100,000 runs -a size too large for previous fully-Bayesian implementations -and demonstrate prediction and UQ superior to that of "big data" competitors. All methods are implemented in the deepgp package on CRAN.

Variational sparse inverse Cholesky approximation for latent Gaussian processes via double Kullback-Leibler minimization

“…We will also explore extensions to deep GPs (cf. Sauer et al, 2022). An implementation of our method, along with code to reproduce all results, will be made publicly available on GitHub.…”

Statistical Deep Learning for Spatial and Spatio-Temporal Data

“…The most common emulator is the Gaussian process emulator (Kennedy & O'Hagan, 2001), where the map between the input parameters and the numerical model outputs is modeled via a Gaussian process. Several developments on the vanilla Gaussian process have been proposed for the purpose of emulation, such as the treed Gaussian process (Gramacy & Lee, 2008) and the deep Gaussian process of Damianou & Lawrence (2013) or variants thereof (Monterrubio-Gómez et al, 2020, Ming et al, 2021, Marmin & Filippone, 2022, Sauer et al, 2022.…”