The ensemble‐adjusted Ignorance Score for forecasts issued as normal distributions

Siegert, Stefan; Ferro, Christopher A. T.; Stephenson, David B.; Leutbecher, Martin

doi:10.1002/qj.3447

Cited by 17 publications

(16 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The actual ensemble size dependence in the NWP ensemble follows reasonably well the analytical expression derived by Siegert et al. s*(). Figure shows the difference in the DSS of an M ‐member ensemble and the infinite‐member ensemble for the same 120 combinations of lead time, variable, and domain shown previously.…”

Section: Convergence Of Probabilistic Skillsupporting

confidence: 81%

“…Siegert et al. s*() determine the expected value of the difference between the DSS of an m ‐member ensemble and the DSS of an infinite‐member ensemble when members are i.i.d. from a normal distribution:

\begin{matrix} E ({DSS}_{m} - {DSS}_{\infty}) = \frac{1}{2} \{ψ (\frac{m - 1}{2}) - \ln (\frac{m - 1}{2})\} \\ + \frac{m - 1}{2 m (m - 3)} + \frac{1}{m - 3} E \frac{{(y - μ)}^{2}}{σ^{2}} . \end{matrix}

The function ψ in Equation is the Digamma function, which is defined as the logarithmic derivative of the Gamma function.…”

Section: Convergence Of Probabilistic Skillmentioning

confidence: 99%

See 1 more Smart Citation

Ensemble size: How suboptimal is less than infinity?

Leutbecher

2018

Quart J Royal Meteoro Soc

Self Cite

View full text Add to dashboard Cite

Ensemble forecasts are the method of choice in numerical weather prediction (NWP) to generate probabilistic forecasts. The number of members in an ensemble is an important factor in determining how well a probability distribution of a weather-related variable can be estimated. Having only a finite number of members reduces the average skill such a probabilistic forecast can have. Increasing ensemble size is therefore desirable; however, ensemble size is also proportional to the computational cost. Having a small ensemble size limits the cost and makes other improvements, such as increases in spatial resolution, feasible. This article examines how average skill measures with metrics such as the continuous ranked probability score, the quantile score, and the Dawid-Sebastiani score converge with ensemble size. A numerical experiment with a 200 member ensemble using the European Centre for Medium-Range Weather Forecasts (ECMWF) Integrated Forecasting System (IFS) model at a resolution of 29 km and a forecast range of 15 days provides data to compare the convergence of probabilistic skill in a current NWP system with theoretical expectations derived for perfectly reliable ensembles with exchangeable members. Results in the first part of the article can help users of operational NWP ensemble forecasts formulate their minimum requirement in terms of ensemble size. In the second part, requirements for scientists who test changes to NWP systems are examined. Using proper scores and fair scores, it is explored whether testing changes in the ensemble forecasts can be meaningful with fewer members than in the operational configuration. Results are based on medium-range numerical experiments with 50 members. Two experiments test the activation of a representation of model uncertainty and three other experiments test changes in horizontal resolution from 29 to 18 km and from 29 to 45 km.

show abstract

Section: Convergence Of Probabilistic Skillsupporting

confidence: 81%

\begin{matrix} E ({DSS}_{m} - {DSS}_{\infty}) = \frac{1}{2} \{ψ (\frac{m - 1}{2}) - \ln (\frac{m - 1}{2})\} \\ + \frac{m - 1}{2 m (m - 3)} + \frac{1}{m - 3} E \frac{{(y - μ)}^{2}}{σ^{2}} . \end{matrix}

The function ψ in Equation is the Digamma function, which is defined as the logarithmic derivative of the Gamma function.…”

Section: Convergence Of Probabilistic Skillmentioning

confidence: 99%

Ensemble size: How suboptimal is less than infinity?

Leutbecher

2018

Quart J Royal Meteoro Soc

Self Cite

View full text Add to dashboard Cite

show abstract

“…3. The green dashed, the blue short-dashed, the pink dots and the light blue dash-dotted curves correspond to the experiments with the 1st to 10th BLVs, with the 11th to 20th BLVs, with the 21st to 30th BLVs and the 1st to 20th BLVs, respectively ◂ index of the ensemble member), the Dawid-Sebastiani Score (DSS) can be written as where 2 i,j is the variance estimator of the ensemble for variable y k,i,j with k = 1, …, K. Corrections for the finite size of the ensemble can also be taken into account (Siegert et al 2019;Leutbecher 2019), but as we are comparing ensembles with equivalent number of members, this does not need to be taken into account. Furthermore, in the following analyses we will also drop the first term of DSS for the same reason.…”

Section: Ensemble Forecasts: Experimental Setupmentioning

confidence: 99%

On the use of near-neutral Backward Lyapunov Vectors to get reliable ensemble forecasts in coupled ocean–atmosphere systems

Vannitsem

Duan

2020

Clim Dyn

View full text Add to dashboard Cite

The use of coupled Backward Lyapunov Vectors (BLV) for ensemble forecast is demonstrated in a coupled ocean-atmosphere system of reduced order, the Modular Arbitrary Order Ocean-Atmosphere Model (MAOOAM). It is found that overall the most suitable BLVs to initialize a (multiscale) coupled ocean-atmosphere forecasting system are the ones associated with near-neutral and slightly negative Lyapunov exponents. This unexpected result is related to the fact that these BLVs display larger projections on the ocean variables than the others, leading to an appropriate spread for the ocean, and at the same time a rapid transfer of these errors toward the most unstable BLVs affecting predominantly the atmosphere is experienced. The latter dynamics is a natural property of any generic perturbation in nonlinear chaotic dynamical systems, allowing for a reliable spread with the atmosphere too. Furthermore, this specific choice becomes even more crucial when the goal is the forecasting of low-frequency variability at annual and decadal time scales. The implications of these results for operational ensemble forecasts in coupled ocean-atmosphere systems are briefly discussed.

show abstract

“…In the presentation of the results we focused on fCRPS. To check that our results are robust and do not change if another probabilistic score is used, we computed the fair ignorance score (fIS: Siegert et al ., 2019). The main conclusions regarding the ranking of experiments do not change when using the fIS instead of fCRPS.…”

Section: Discussionmentioning

confidence: 99%

Revision of the Stochastically Perturbed Parametrisations model uncertainty scheme in the Integrated Forecasting System

Lang

Lock

Leutbecher

et al. 2021

Quart J Royal Meteoro Soc

Self Cite

View full text Add to dashboard Cite

The Stochastically Perturbed Parametrisations scheme (SPP) represents model uncertainty in numerical weather prediction by introducing stochastic perturbations into the physical parametrisation schemes. The perturbations are constructed in such a way that the internal consistency of the physical parametrisation schemes is preserved. We developed a revised version of SPP for the Integrated Forecasting System of the European Centre for Medium‐Range Weather Forecasts (ECMWF). The revised version introduces perturbations to additional quantities and modifies the probability distributions sampled by the scheme. Medium‐range ensemble forecasts with the revised SPP are considerably more skilful than ensemble forecasts with the original implementation of SPP. The revised version of SPP is similar, in terms of forecast skill, to the Stochastically Perturbed Parametrisation Tendency scheme (SPPT), which is currently used to represent model uncertainty in the operational ECMWF ensemble forecasts.

show abstract

The ensemble‐adjusted Ignorance Score for forecasts issued as normal distributions

Cited by 17 publications

References 41 publications

Ensemble size: How suboptimal is less than infinity?

Ensemble size: How suboptimal is less than infinity?

On the use of near-neutral Backward Lyapunov Vectors to get reliable ensemble forecasts in coupled ocean–atmosphere systems

Revision of the Stochastically Perturbed Parametrisations model uncertainty scheme in the Integrated Forecasting System

Contact Info

Product

Resources

About