Neighborhood-Based Ensemble Evaluation Using the CRPS

Abstract

The neighborhood-based ensemble evaluation using the Continuous Ranked Probability Score is based on the pooling of the Cumulative Density Function (CDF) for all the points inside a neighborhood. This methodology can be applied to the forecast CDF for measuring the predictive input of neighboring points in the center of the neighborhood. It can also be applied at the same time to forecast CDF and observed CDF so as to quantify the quality of the pooled ensemble forecast at the scale of the neighborhood. Fair versions of these two neighborhood scores are also defined in order to reduce their dependencies on the size of ensemble forecasts. The borderline case of deterministic forecasts is also explored so as to be able to compare them with ensemble forecasts. The information of these new scores is analyzed on idealized and real cases of rain accumulated during three hours and of 2-m temperature forecast by four deterministic and probabilistic forecasting systems operational at Météo-France.

Comment on egusphere-2024-1277

“…The performance of the SVS ensemble for simulating soil moisture, soil temperature, snow depth, and deep percolation was evaluated using the Continuous Ranked Probability Score (CRPS, Grimit et al (2006); Matheson and Winkler (1976)). We used the formulation proposed by Stein and Stoop (2022) which evaluates the distance between the simulation CDF (F) and the observation CDF (G) based on the following formulation (Baringhaus and Franz, 2004;Székely and Rizzo, 2005):…”

“…where X, X ′ (Y , Y ′ ) represent independent copies of a random variable with a CDF given by F (G). For a given time step, we can calculate the CRPS of the SVS ensemble by (Stein and Stoop, 2022):…”

Cold Climates, Complex Hydrology: Can A Land Surface Model Accurately Simulate Deep Percolation?

“…The performance of the SVS ensemble for simulating soil moisture, soil temperature, snow depth, and deep percolation was evaluated using the Continuous Ranked Probability Score (CRPS, Grimit et al (2006); Matheson and Winkler (1976)). We used the formulation proposed by Stein and Stoop (2022) which evaluates the distance between the simulation CDF (F) and the observation CDF (G) based on the following formulation (Baringhaus and Franz, 2004;Székely and Rizzo, 2005):…”

“…where X, X ′ (Y , Y ′ ) represent independent copies of a random variable with a CDF given by F (G). For a given time step, we can calculate the CRPS of the SVS ensemble by (Stein and Stoop, 2022):…”