Ensembles of leading European global coupled climate models show impressive reliability for seasonal climate prediction-including useful output for probabilistic prediction of malaria incidence and crop yield.

The DEMETER multi‐model ensemble system is used to investigate the rationale behind the multi‐model concept. A comprehensive documentation of the differences in the single and multi‐model performance in the DEMETER hindcast data set is given. Both deterministic and probabilistic diagnostics are used and a variety of analyses demonstrate the improvements achieved by using multi‐model instead of single‐model ensembles. In order to understand the reason behind the multi‐model superiority, basic scenarios describing how the multi‐model approach can improve over single‐model skill are discussed. It is demonstrated that multi‐model superiority is caused not only by error compensation but in particular by its greater consistency and reliability.

The control of epidemic malaria is a priority for the international health community and specific targets for the early detection and effective control of epidemics have been agreed. Interannual climate variability is an important determinant of epidemics in parts of Africa where climate drives both mosquito vector dynamics and parasite development rates. Hence, skilful seasonal climate forecasts may provide early warning of changes of risk in epidemic-prone regions. Here we discuss the development of a system to forecast probabilities of anomalously high and low malaria incidence with dynamically based, seasonal-timescale, multi-model ensemble predictions of climate, using leading global coupled ocean-atmosphere climate models developed in Europe. This forecast system is successfully applied to the prediction of malaria risk in Botswana, where links between malaria and climate variability are well established, adding up to four months lead time over malaria warnings issued with observed precipitation and having a comparably high level of probabilistic prediction skill. In years in which the forecast probability distribution is different from that of climatology, malaria decision-makers can use this information for improved resource allocation.

No abstract

▪ Abstract  Weather and climate predictions are uncertain, because both forecast initial conditions and the computational representation of the known equations of motion are uncertain. Ensemble prediction systems provide the means to estimate the flow-dependent growth of uncertainty during a forecast. Sources of uncertainty must therefore be represented in such systems. In this paper, methods used to represent model uncertainty are discussed. It is argued that multimodel and related ensembles are vastly superior to corresponding single-model ensembles, but do not provide a comprehensive representation of model uncertainty. A relatively new paradigm is discussed, whereby unresolved processes are represented by computationally efficient stochastic-dynamic schemes.

The DEMETER multi‐model ensemble system is used to investigate the enhancement in seasonal predictability that can be achieved by calibrating single‐model ensembles and combining them to issue multi‐model predictions. The forecast quality of both deterministic and probabilistic predictions is assessed and compared to the skill of a simple multi‐model ensemble where all the single models are equally weighted. Both calibration and combination are carried out using cross‐validation. Single‐model seasonal ensembles are calibrated using canonical correlation analysis for model adjustment and variance inflation for reliability enhancement. Results indicate that both model adjustment and inflation increase the skill of tropical predictions for single‐model ensembles, provided that the training time series are long enough. Some improvements are also found for extratropical areas, although mostly due to an increase of reliability associated with the inflation. The beneficial impact of calibration is smaller for the simple multi‐model than for the single‐model ensembles due to the relatively high reliability of the former. The raw single‐model predictions are also linearly combined using grid‐point multiple linear regression to create an optimized multi‐model system. Results indicate that the forecast quality of the simple multi‐model ensemble is generally difficult to improve using multiple linear regression due to the lack of robustness of the regression coefficients. As in the case of the calibration, longer time series would be preferred to achieve a significant forecast quality improvement. Over the tropics, a multiple linear regression, that uses the principal components of the model anomalies for the target area as predictors indicates a substantial gain in skill even with the available sample size. The implications of these results in an operational context are discussed.

Recently, the European Centre for Medium-Range Weather Forecasts (ECMWF) produced a reforecast dataset for a 2005 version of their ensemble forecast system. The dataset consisted of 15-member reforecasts conducted for the 20-yr period 1982–2001, with reforecasts computed once weekly from 1 September to 1 December. This dataset was less robust than the daily reforecast dataset produced for the National Centers for Environmental Prediction (NCEP) Global Forecast System (GFS), but it utilized a much higher-resolution, more recent model. This manuscript considers the calibration of 2-m temperature forecasts using these reforecast datasets as well as samples of the last 30 days of training data. Nonhomogeneous Gaussian regression was used to calibrate forecasts at stations distributed across much of North America. Significant observations included the following: (i) although the “raw” GFS forecasts (probabilities estimated from ensemble relative frequency) were commonly unskillful as measured in continuous ranked probability skill score (CRPSS), after calibration with a 20-yr set of weekly reforecasts their skill exceeded that of the raw ECMWF forecasts; (ii) statistical calibration using the 20-yr weekly ECMWF reforecast dataset produced a large improvement relative to the raw ECMWF forecasts, such that the ∼4–5-day calibrated reforecast-based product had a CRPSS as large as a 1-day raw forecast; (iii) a calibrated multimodel GFS/ECMWF forecast trained on 20-yr weekly reforecasts was slightly more skillful than either the individual calibrated GFS or ECMWF reforecast products; (iv) approximately 60%–80% of the improvement from calibration resulted from the simple correction of time-averaged bias; (v) improvements were generally larger at locations where the forecast skill was originally lower, and these locations were commonly found in regions of complex terrain; (vi) the past 30 days of forecasts were adequate as a training dataset for short-lead forecasts, but longer-lead forecasts benefited from more training data; and (vii) a small but consistent improvement was produced by calibrating GFS forecasts using the full 25-yr, daily reforecast training dataset versus the subsampled, 20-yr weekly training dataset.

As a companion to Part I, which discussed the calibration of probabilistic 2-m temperature forecasts using large training datasets, Part II discusses the calibration of probabilistic forecasts of 12-hourly precipitation amounts. Again, large ensemble reforecast datasets from the European Centre for Medium-Range Weather Forecasts (ECMWF) and the Global Forecast System (GFS) were used for testing and calibration. North American Regional Reanalysis (NARR) 12-hourly precipitation analysis data were used for verification and training. Logistic regression was used to perform the calibration, with power-transformed ensemble means and spreads as predictors. Forecasts were produced and validated for every NARR grid point in the conterminous United States (CONUS). Training sample sizes were increased by including data from 10 nearby grid points with similar analyzed climatologies. “Raw” probabilistic forecasts from each system were considered, in which probabilities were set according to ensemble relative frequency. Calibrated forecasts were also considered based on three amounts of training data: the last 30 days of forecasts (available for 2005 only), weekly reforecasts during 1982–2001, and daily reforecasts during 1979–2003 (GFS only). Several main results were found. (i) Raw probabilistic forecasts from the ensemble prediction systems’ relative frequency possessed little or negative skill when skill was computed with a version of the Brier skill score (BSS) that does not award skill solely on the basis of differences in climatological probabilities among samples. ECMWF raw forecasts had larger skills than GFS raw forecasts. (ii) After calibration with weekly reforecasts, ECMWF forecasts were much improved in reliability and were moderately skillful. Similarly, GFS-calibrated forecasts were much more reliable, albeit somewhat less skillful. Nonetheless, GFS-calibrated forecasts were much more skillful than ECMWF raw forecasts. (iii) The last 30 days of training data produced calibrated forecasts of light-precipitation events that were nearly as skillful as those with weekly reforecast data. However, for higher precipitation thresholds, calibrated forecasts using the weekly reforecast datasets were much more skillful, indicating the importance of large sample size for the calibration of unusual and rare events. (iv) Training with daily GFS reforecast data provided calibrated forecasts with a skill only slightly improved relative to that from the weekly data.