Probabilistic Forecast Calibration Using ECMWF and GFS Ensemble Reforecasts. Part I: Two-Meter Temperatures

Abstract

Recently, the European Centre for Medium-Range Weather Forecasts (ECMWF) produced a reforecast dataset for a 2005 version of their ensemble forecast system. The dataset consisted of 15-member reforecasts conducted for the 20-yr period 1982–2001, with reforecasts computed once weekly from 1 September to 1 December. This dataset was less robust than the daily reforecast dataset produced for the National Centers for Environmental Prediction (NCEP) Global Forecast System (GFS), but it utilized a much higher-resolution, more recent model. This manuscript considers the calibration of 2-m temperature forecasts using these reforecast datasets as well as samples of the last 30 days of training data. Nonhomogeneous Gaussian regression was used to calibrate forecasts at stations distributed across much of North America. Significant observations included the following: (i) although the “raw” GFS forecasts (probabilities estimated from ensemble relative frequency) were commonly unskillful as measured in continuous ranked probability skill score (CRPSS), after calibration with a 20-yr set of weekly reforecasts their skill exceeded that of the raw ECMWF forecasts; (ii) statistical calibration using the 20-yr weekly ECMWF reforecast dataset produced a large improvement relative to the raw ECMWF forecasts, such that the ∼4–5-day calibrated reforecast-based product had a CRPSS as large as a 1-day raw forecast; (iii) a calibrated multimodel GFS/ECMWF forecast trained on 20-yr weekly reforecasts was slightly more skillful than either the individual calibrated GFS or ECMWF reforecast products; (iv) approximately 60%–80% of the improvement from calibration resulted from the simple correction of time-averaged bias; (v) improvements were generally larger at locations where the forecast skill was originally lower, and these locations were commonly found in regions of complex terrain; (vi) the past 30 days of forecasts were adequate as a training dataset for short-lead forecasts, but longer-lead forecasts benefited from more training data; and (vii) a small but consistent improvement was produced by calibrating GFS forecasts using the full 25-yr, daily reforecast training dataset versus the subsampled, 20-yr weekly training dataset.

On the skill of various ensemble spread estimators for probabilistic short range wind forecasting

“…The EPS dyn is further calibrated by applying a cut-off variant of the nonhomogeneous Gaussian regression (NGR) technique (Thorarinsdottir and Gneiting, 2010;Gneiting et al, 2005;Hagedorn et al, 2008;Kann et al, 2009). This technique statistically calibrates the mean and the ensemble variance by minimizing the Continuous Ranked Probability Score (CRPS) within a certain training period.…”

“…The latter two methods are characterized by the fact that low as well as high observed frequencies are overforecast (apart from the very low frequencies which are underforecast). The Continuous Ranked Probability Score (CRPS), which can be decomposed into a reliability, resolution and uncertainty term, is the integral form of the discrete ranked probability score over all (possible) thresholds (Hersbach, 2000). It compares a full distribution with the observation, where both are represented as cumulative distribution functions (CDFs).…”

Summary of recommendations of the first workshop on Postprocessing and Downscaling Atmospheric Forecasts for Hydrologic Applications held at Météo‐France, Toulouse, France, 15–18 June 2009

“…Hamill et al (2004) and Hamill and Whitaker (2006) have declared reforecasting as an efficient means of calibrating EPS and argue that the benefits of re-forecasts are so large that they should become an integral part of the NWP process. Hagedorn et al (2007) point out that a growing body of literature indicates the potential utility of re-forecast methodology for improving operational ensemble predictions. Unfortunately, re-forecasts are costly to produce and are not systematically available for all NWP EPS.…”

A unified linear Model Output Statistics scheme for both deterministic and ensemble forecasts

“…The results are compared with a technique widely used for ensemble post-processing, the NGR method (e.g. Gneiting et al, 2005;Wilks, 2006a;Hagedorn et al, 2008), in order to evaluate the ability of this scheme to compete with other post-processing techniques. This technique is based on the assumption that the forecast error is approximately Gaussian and the calibration of the fitted distribution is performed through the modification of the mean and variance of the original ensemble.…”

“…The aim is to empirically modify the ensemble probability distribution based on the knowledge acquired on the behaviour of past ensemble forecasts. These approaches have been shown to provide important improvements of the ensemble forecasts (Hamill and Collucci, 1998;Roulston and Smith, 2003;Gneiting et al, 2005;Stephenson et al, 2005;Wang and Bishop, 2005;Hagedorn et al, 2008).…”