Probabilistic Forecasting with Generative Networks via Scoring Rule Minimization

Abstract

Probabilistic forecasting consists of stating a probability distribution for a future outcome based on past observations. In meteorology, ensembles of physics-based numerical models are run to get such distribution. Usually, performance is evaluated with scoring rules, functions of the forecast distribution and the observed outcome. With some scoring rules, calibration and sharpness of the forecast can be assessed at the same time.In deep learning, generative neural networks parametrize distributions on high-dimensional spaces and easily allow sampling by transforming draws from a latent variable. Conditional generative networks additionally constrain the distribution on an input variable. In this manuscript, we perform probabilistic forecasting with conditional generative networks trained to minimize scoring rule values. In contrast to Generative Adversarial Networks (GANs), no discriminator is required and training is stable. We perform experiments on two chaotic models and a global dataset of weather observations; results are satisfactory and better calibrated than what achieved by GANs.

Likelihood-Free Inference with Generative Neural Networks via Scoring Rule Minimization

“…In simulation studies, and especially on high-dimensional tasks, we found the Scoring Rule approach generally performed better and was substantially cheaper to train. These findings corroborate similar ones reported in Pacchiardi et al [2022] in the setting of probabilistic forecasting, making Scoring Rules minimization an appealing method to train generative networks, particularly when uncertainty quantification in the approximate distribution is critical.…”

“…We discuss here how to use Scoring Rules to define an adversarial-free training objective for generative networks, focusing on the specific case of a generative network parametrizing an approximate posterior. In Pacchiardi et al [2022], more details on SR-training and its application to probabilistic forecasting can be found. Other works employing SR training, albeit not for the LFI framework, are Bouchacourt et al [2016], Gritsenko et al [2020], Harakeh and Waslander [2021].…”

“…A way to encode some of it is to compute the SRs on localized patches across the grid and cumulate the score; in this way, short-scale correlations are given more importance. However, the resulting Scoring Rule is not strictly proper; to fix this, we add the SR computed over the full x, which makes the overall SR strictly proper (see Lemma 3.4 in Pacchiardi et al [2022]).…”

“…where w 1 , w 2 > 0, | p denotes the restriction of a distribution or of a vector to a patch p and P is a set of patches. See Pacchiardi et al [2022] for more discussion on patched SRs.…”

“…Here, we build on Ramesh et al [2022] by proposing a different training strategy based on minimizing values of Scoring Rules (SR, , which are functions assessing the match between a probability distribution and an observation. In contrast to the adversarial approach, plagued by biased gradient estimates [Bińkowski et al, 2018] and often leading to mode collapse (in which the parametrized probability distribution collapses on a single point, Richardson and Weiss, 2018), SR training has been found to better capture the full shape of the probability distribution in the setting of probabilistic forecasting [Pacchiardi et al, 2022]. Additionally, the SR-minimization approach leads to simpler training with respect to the adversarial one, as it does not require a discriminator or solving a min-max problem.…”