Parameter uncertainty in forecast recalibration

Abstract

Ensemble forecasts of weather and climate are subject to systematic biases in the ensemble mean and variance, leading to inaccurate estimates of the forecast mean and variance. To address these biases, ensemble forecasts are post-processed using statistical recalibration frameworks. These frameworks often specify parametric probability distributions for the verifying observations. A common choice is the normal distribution with mean and variance specified by linear functions of the ensemble mean and variance. The parameters of the recalibration framework are estimated from historical archives of forecasts and verifying observations. Often there are relatively few forecasts and observations available for parameter estimation, and so the fitted parameters are also subject to uncertainty. This artefact is usually ignored. This study reviews analytic results that account for parameter uncertainty in the widely used Model Output Statistics recalibration framework. The predictive bootstrap is used to approximate the parameter uncertainty by resampling in more general frameworks such as non-homogeneous Gaussian regression. Forecasts on daily, seasonal and annual time-scales are used to demonstrate that accounting for parameter uncertainty in the recalibrated predictive distributions leads to probability forecasts that are more skilful and reliable than those in which parameter uncertainty is ignored. The improvements are attributed to more reliable tail probabilities of the recalibrated forecast distributions.

Parametric decadal climate forecast recalibration (DeFoReSt 1.0)

“…Furthermore, parameter uncertainty due to a small training size may result in forecasts that are still underdispersive after recalibration. For the seasonal scale, this has been discussed by Siegert et al (2015). However, for decadal climate forecasts, this aspect should be further considered in future studies.…”

“…(6) would also be a step function. The CRPS can therefore be interpreted as a distance measure between the probabilistic forecast and the verifying observation (Siegert et al, 2015). The continuous ranked probability skill score (CRPSS) is, as the name implies, the corresponding skill score.…”

Best Practices for Postprocessing Ensemble Climate Forecasts. Part I: Selecting Appropriate Recalibration Methods

“…The effect of uncertainty in the recalibration parameters is also routinely ignored. This issue has recently been addressed for regression frameworks, such as the one presented here, by Siegert et al (2016). However, additional development is still required to account for observational uncertainty during recalibration.…”

Enforcing calibration in ensemble postprocessing

“…However, calibration is not guaranteed because the CRPS minimization may be achieved as a compromise between calibration and resolution (which is related to sharpness), so that the raw ensemble miscalibration may not be fully corrected (e.g. Gneiting et al, 2005;Thorarinsdottir and Gneiting, 2010;Lerch and Thorarinsdottir, 2013;Möller and Gross, 2016;Siegert et al, 2016;Williams, 2016). Estimating ensemble postprocessing parameters by maximizing likelihood can also yield imperfectly corrected ensemble calibration (e.g.…”