A Comparison between Raw Ensemble Output, (Modified) Bayesian Model Averaging, and Extended Logistic Regression Using ECMWF Ensemble Precipitation Reforecasts

Abstract

Using a 20-yr ECMWF ensemble reforecast dataset of total precipitation and a 20-yr dataset of a dense precipitation observation network in the Netherlands, a comparison is made between the raw ensemble output, Bayesian model averaging (BMA), and extended logistic regression (LR). A previous study indicated that BMA and conventional LR are successful in calibrating multimodel ensemble forecasts of precipitation for a single forecast projection. However, a more elaborate comparison between these methods has not yet been made. This study compares the raw ensemble output, BMA, and extended LR for single-model ensemble reforecasts of precipitation; namely, from the ECMWF ensemble prediction system (EPS). The raw EPS output turns out to be generally well calibrated up to 6 forecast days, if compared to the area-mean 24-h precipitation sum. Surprisingly, BMA is less skillful than the raw EPS output from forecast day 3 onward. This is due to the bias correction in BMA, which applies model output statistics to individual ensemble members. As a result, the spread of the bias-corrected ensemble members is decreased, especially for the longer forecast projections. Here, an additive bias correction is applied instead and the equation for the probability of precipitation in BMA is also changed. These modifications to BMA are referred to as ''modified BMA'' and lead to a significant improvement in the skill of BMA for the longer projections. If the area-maximum 24-h precipitation sum is used as a predictand, both modified BMA and extended LR improve the raw EPS output significantly for the first 5 forecast days. However, the difference in skill between modified BMA and extended LR does not seem to be statistically significant. Yet, extended LR might be preferred, because incorporating predictors that are different from the predictand is straightforward, in contrast to BMA.

Geostatistical Model Averaging for Locally Calibrated Probabilistic Quantitative Precipitation Forecasting

“…Recently, Wilks (2009) proposed an appealing development that avoids inconsistencies between logistic regression equations at distinct thresholds and results in full predictive distributions. Schmeits and Kok (2010) compared his approach to BMA, with the methods performing similarly. The basic idea of GMA, namely locally varying model parameters, can be implemented in the logistic regression framework as well.…”

Towards Improved Understanding of the Applicability of Uncertainty Forecasts in the Electric Power Industry

“…In this way the correlation between the two variables is also determined, resulting in more physically coherent wind speed and direction forecasts. Despite significant differences in the proposed statistical modeling approaches, several comparative studies conclude that the underlying training database determines the improvement over the raw ensemble weather forecast rather than the choice of the post-processing technique [57,68]. Therefore, it is the specific application's objective that determines the required type of weather forecast product and respective suitable statistical post-processing method.…”

Calibrating Weather Forecast using Bayesian Model Averaging and Geostatistical Output Perturbation

“…where k is the number of pair of distance between two locations and 1 = 1 (1), 1 (2), is the distance which represents the whole pair of two locations being involved [4]. The next step is carrying out the estimation of those three parameters based on objective function given on Eq.…”

“…Hence, statistical postprocessing needs to be applied to NWP output by using ensemble, such as combination of NWPs from several meteorological authorities. Though in many cases, ensemble forecast still possesses underdispersive nature, that is the forecast tends to concentrate at a point with low variance causing the observation outside the predictive interval, then as a consequence they need to be calibrated [2]. In order to handle such case, BMA and GOP could be applied to calibrate the ensemble forecast, among others.…”