Verification and Calibration of Neighborhood and Object-Based Probabilistic Precipitation Forecasts from a Multimodel Convection-Allowing Ensemble

Abstract

Neighborhood and object-based probabilistic precipitation forecasts from a convection-allowing ensemble are verified and calibrated. Calibration methods include logistic regression, one- and two-parameter reliability-based calibration, and cumulative distribution function (CDF)-based bias adjustment. Newly proposed object-based probabilistic forecasts for the occurrence of a forecast object are derived from the percentage of ensemble members with a matching object. Verification and calibration of single- and multimodel subensembles are performed to explore the effect of using multiple models. The uncalibrated neighborhood-based probabilistic forecasts have skill minima during the afternoon convective maximum. Calibration generally improves the skill, especially during the skill minima, resulting in positive skill. In general all calibration methods perform similarly, with a slight advantage of logistic regression (one-parameter reliability based) calibration for 1-h (6 h) accumulations. The uncalibrated object-based probabilistic forecasts are, in general, less skillful than the uncalibrated neighborhood-based probabilistic forecasts. Object-based calibration also results in positive skill at all lead times. For object-based calibration the skill is significantly different among the calibration methods, with the logistic regression performing the best and CDF-based bias adjustment performing the worst. For both the neighborhood and object-based probabilistic forecasts, the impact of using 10 or 25 days of training data for calibration is generally small and is most significant for the two-parameter reliability-based method. An uncalibrated Advanced Research Weather Research and Forecasting Model (ARW-WRF) subensemble is significantly more skillful than an uncalibrated WRF Nonhydrostatic Mesoscale Model (NMM) subensemble. The difference is reduced by calibration. The multimodel subensemble only shows an advantage for the neighborhood-based forecasts beyond 1-day lead time and shows no advantage for the object-based forecasts.

Simulating North American mesoscale convective systems with a convection-permitting climate model

“…In the Southeast and the MidAtlantic region, MCS frequency is up to 70 % overestimated during JJA. Similar overestimations are found in convectionpermitting weather forecasting models in all US regions (Johnson and Wang 2012;Clark et al 2014). Different from forecasting biases are the 50 % underestimation of MCSs in the central US during late summer.…”

A new method for the characterization and verification of local spatial predictability for convective‐scale ensembles

“…Several more suitable methods have been proposed for verifying deterministic forecasts (e.g. Ebert, ; Gilleland et al , ; Johnson and Wang, ) that can now be developed for convection‐permitting ensembles.…”

Assessing spatial precipitation uncertainties in a convective‐scale ensemble

“…To address the double penalty problem in the verification of deterministic precipitation forecasts, a number of new forecast performance metrics have been developed (e.g. Roberts and Lean, ; Ebert, ; Gilleland et al , ; Johnson and Wang, ). More recently, new methods have been explored for characterising both the skill and dispersion of convective‐scale ensemble forecasts (Clark et al ; Johnson et al ; Surcel et al ; Dey et al ).…”