Practical Conditional Neural Processes Via Tractable Dependent Predictions

Abstract

Conditional Neural Processes (CNPs; Garnelo et al., 2018a) are meta-learning models which leverage the flexibility of deep learning to produce well-calibrated predictions and naturally handle off-the-grid and missing data. CNPs scale to large datasets and train with ease. Due to these features, CNPs appear well-suited to tasks from environmental sciences or healthcare. Unfortunately, CNPs do not produce correlated predictions, making them fundamentally inappropriate for many estimation and decision making tasks. Predicting heat waves or floods, for example, requires modelling dependencies in temperature or precipitation over time and space. Existing approaches which model output dependencies, such as Neural Processes (NPs; Garnelo et al., 2018b) or the FullConvGNP (Bruinsma et al., 2021), are either complicated to train or prohibitively expensive. What is needed is an approach which provides dependent predictions, but is simple to train and computationally tractable. In this work, we present a new class of Neural Process models that make correlated predictions and support exact maximum likelihood training that is simple and scalable. We extend the proposed models by using invertible output transformations, to capture non-Gaussian output distributions. Our models can be used in downstream estimation tasks which require dependent function samples. By accounting for output dependencies, our models show improved predictive performance on a range of experiments with synthetic and real data.

Adversarially Contrastive Estimation of Conditional Neural Processes

“…Combining this with tractable distribution transformation approaches, such as normalizing flows (NFs), may provide a solution. It is not trivial since NFs could challenge the consistency of CNPs as discussed in (Markou et al, 2022). Another solution might be applying adversarial training to latent NPs, since they have a more flexible predictive space than CNPs.…”

“…Having said that, NCE suggests a principle for training EBMs to estimate an unknown distribution by discriminating between data drawn therefrom and noises from a known distribution. In this context, CNPs are ideal noise generators provided that 1) samples can be easily obtained by viewing them as prediction maps (Markou et al, 2022), and 2) density function is analytically tractable and exact.…”

Reply on RC2

“…Finally, we investigate current state-of-the-art downscaling models: Convolutional Conditional Neural Processes (ConvCNP) (Gordon et al, 2019;Vaughan et al, 2021) and Convolutional Gaussian Neural Processes (ConvGNP) (Markou et al, 2022;Andersson et al, 2023). These models offer similar advantages to the MFGP model, including: capturing extreme values and spatiotemporal structure, generalising to multiple locations, predicting at arbitrary locations and overcoming gridding biases.…”

Downscaling precipitation over High Mountain Asia using Multi-Fidelity Gaussian Processes: Improved estimates from ERA5

“…Finally, we investigate current state-of-the-art downscaling models: Convolutional Conditional Neural Processes (ConvCNP) (Gordon et al, 2019;Vaughan et al, 2021) and Convolutional Gaussian Neural Processes (ConvGNP) (Markou et al, 2022;Andersson et al, 2023). These models offer similar advantages to the MFGP model, including: capturing extreme values and spatiotemporal structure, generalising to multiple locations, predicting at arbitrary locations and overcoming gridding biases.…”