The present paper summarizes the methodologies used at the European Centre for Medium-Range Weather Forecasts (ECMWF), the Meteorological Service of Canada (MSC), and the National Centers for Environmental Prediction (NCEP) to simulate the effect of initial and model uncertainties in ensemble forecasting. The characteristics of the three systems are compared for a 3-month period between May and July 2002. The main conclusions of the study are the following:the performance of ensemble prediction systems strongly depends on the quality of the data assimilation system used to create the unperturbed (best) initial condition and the numerical model used to generate the forecasts;a successful ensemble prediction system should simulate the effect of both initial and model-related uncertainties on forecast errors; andfor all three global systems, the spread of ensemble forecasts is insufficient to systematically capture reality, suggesting that none of them is able to simulate all sources of forecast uncertainty.The relative strengths and weaknesses of the three systems identified in this study can offer guidelines for the future development of ensemble forecasting techniques.

An ensemble Kalman filter (EnKF) has been implemented for atmospheric data assimilation. It assimilates observations from a fairly complete observational network with a forecast model that includes a standard operational set of physical parameterizations. To obtain reasonable results with a limited number of ensemble members, severe horizontal and vertical covariance localizations have been used. It is observed that the error growth in the data assimilation cycle is mainly due to model error. An isotropic parameterization, similar to the forecast-error parameterization in variational algorithms, is used to represent model error. After some adjustment, it is possible to obtain innovation statistics that agree with the ensemble-based estimate of the innovation amplitudes for winds and temperature. Currently, no model error is added for the humidity variable, and, consequently, the ensemble spread for humidity is too small. After about 5 days of cycling, fairly stable global filter statistics are obtained with no sign of filter divergence. The quality of the ensemble mean background field, as verified using radiosonde observations, is similar to that obtained using a 3D variational procedure. In part, this is likely due to the form chosen for the parameterized model error. Nevertheless, the degree of similarity is surprising given that the background-error statistics used by the two procedures are rather different, with generally larger background errors being used by the variational scheme. A set of 5-day integrations has been started from the ensemble of initial conditions provided by the EnKF. For the middle and lower troposphere, the growth rates of the perturbations are somewhat smaller than the growth rate of the actual ensemble mean error. For the upper levels, the perturbation patterns decay for about 3 days as a consequence of diffusive model dynamics. These decaying perturbations tend to severely underestimate the actual error that grows rapidly near the model top.

The ensemble Kalman filter (EnKF) has been proposed for operational atmospheric data assimilation. Some outstanding issues relate to the required ensemble size, the impact of localization methods on balance, and the representation of model error.To investigate these issues, a sequential EnKF has been used to assimilate simulated radiosonde, satellite thickness, and aircraft reports into a dry, global, primitive-equation model. The model uses the simple forcing and dissipation proposed by Held and Suarez. It has 21 levels in the vertical, includes topography, and uses a 144 ϫ 72 horizontal grid. In total, about 80 000 observations are assimilated per day.It is found that the use of severe localization in the EnKF causes substantial imbalance in the analyses. As the distance of imposed zero correlation increases to about 3000 km, the amount of imbalance becomes acceptably small.A series of 14-day data assimilation cycles are performed with different configurations of the EnKF. Included is an experiment in which the model is assumed to be perfect and experiments in which model error is simulated by the addition of an ensemble of approximately balanced model perturbations with a specified statistical structure.The results indicate that the EnKF, with 64 ensemble members, performs well in the present context.The growth rate of small perturbations in the model is examined and found to be slow compared with the corresponding growth rate in an operational forecast model. This is partly due to a lack of horizontal resolution and partly due to a lack of realistic parameterizations. The growth rates in both models are found to be smaller than the growth rate of differences between forecasts with the operational model and verifying analyses. It is concluded that model-error simulation would be important, if either of these models were to be used with the EnKF for the assimilation of real observations.

An updated global ensemble prediction system became operational at the Meteorological Service of Canada in July 2007. The new elements of the system include the use of 20 members instead of 16, a single dynamical core [the Global Environmental Multiscale (GEM) model], stochastic physical tendency perturbations and a kinetic energy backscatter algorithm, an ensemble Kalman filter with four-dimensional data handling, and a decrease from 1.28 to 0.98 in horizontal grid spacing. This system is compared with the former operational one using a variety of probabilistic measures. For global upper-air dynamical fields, the improvement in predictive skill for equivalent forecast quality is from 9 to 16 h around day 6. Precipitation forecasts, verified over Canada, are also significantly improved. The impact of each of the abovementioned new elements of the ensemble prediction system is also evaluated separately in a series of sensitivity experiments for which one given element is removed from the system.

A verification system has been developed for the ensemble prediction system (EPS) at the Canadian Meteorological Centre (CMC). This provides objective criteria for comparing two EPSs, necessary when deciding whether or not to implement a new or revised EPS. The proposed verification methodology is based on the continuous ranked probability score (CRPS), which provides an evaluation of the global skill of an EPS. Its reliability/resolution partition, proposed by Hersbach, is used to measure the two main attributes of a probabilistic system. Also, the characteristics of the reliability are obtained from the two first moments of the reduced centered random variable (RCRV), which define the bias and the dispersion of an EPS. Resampling bootstrap techniques have been applied to these scores. Confidence intervals are thus defined, expressing the uncertainty due to the finiteness of the number of realizations used to compute the scores. All verifications are performed against observations to provide more independent validations and to avoid any local systematic bias of an analysis. A revised EPS, which has been tested at the CMC in a parallel run during the autumn of 2005, is described in this paper. This EPS has been compared with the previously operational one with the verification system presented above. To illustrate the verification methodology, results are shown for the temperature at 850 hPa. The confidence intervals are computed by taking into account the spatial correlation of the data and the temporal autocorrelation of the forecast error. The revised EPS performs significantly better for all the forecast ranges, except for the resolution component of the CRPS where the improvement is no longer significant from day 7. The significant improvement of the reliability is mainly due to a better dispersion of the ensemble. Finally, the verification system correctly indicates that variations are not significant when two theoretically similar EPSs are compared.