Probabilistic Verification of Storm Prediction Center Convective Outlooks

Abstract

Eight years’ worth of day 1 and 4.5 years’ worth of day 2–3 probabilistic convective outlooks from the Storm Prediction Center (SPC) are converted to probability grids spanning the continental United States (CONUS). These results are then evaluated using standard probabilistic forecast metrics including the Brier skill score and reliability diagrams. Forecasts are gridded in two different ways: one with a high-resolution grid and interpolation between probability contours and another on an 80-km-spaced grid without interpolation. Overall, the highest skill is found for severe wind forecasts and the lowest skill is observed for tornadoes; for significant severe criteria, the opposite discrepancy is observed, with highest forecast skill for significant tornadoes and approximately no overall forecast skill for significant severe winds. Highest climatology-relative skill is generally observed over the central and northern Great Plains and Midwest, with the lowest—and often negative—skill seen in the West, southern Texas, and the Atlantic Southeast. No discernible year-to-year trend in skill was identified; seasonally, forecasts verified the best in the spring and late autumn and worst in the summer and early autumn. Forecasts are also evaluated in CAPE-versus-shear parameter space; forecasts struggle most in very low shear but also in high-shear, low-CAPE environments. In aggregate, forecasts for all variables verified more skillfully using interpolated probability grids, suggesting utility in interpreting forecasts as a continuous field. Forecast reliability results depend substantially on the interpretation of the forecast fields, but day 1 and day 2–3 tornado outlooks consistently exhibit an underforecast bias.

Forecasting Severe Weather with Random Forests

“…For Day 4 and beyond, only 15% and 30% contours are issued, and for significant severe risk, only a single 10% contour is drawn. For more information on SPC's forecasting process, including historical changes to severe weather and product definitions, see Hitchens and Brooks (2014), Edwards et al (2015), or Herman et al (2018).…”

“…Hitchens and Brooks (2012) investigated the skill of Day 1 categorical outlooks, and this effort was expanded to include evaluation of Days 2 and 3-among other additions-in Hitchens and Brooks (2014). Early published efforts to verify SPC's convective outlooks probabilistically (e.g., Kay and Brooks 2000) have received renewed attention in Hitchens and Brooks (2017) and more formally in Herman et al (2018). Collectively, these studies have demonstrated improving skill in short-to medium-range severe weather forecasts in association with improved numerical weather prediction (NWP), though advances have been stagnating somewhat in recent years.…”

“…As noted above, SPC's convective outlooks are based on only a finite set of probability contours, producing discontinuous jumps in gridded probability fields. Herman et al (2018) demonstrated that forecast skill is improved, albeit not uniformly, when probabilities are interpreted as interpolated between human-drawn probability contours. In these interpolated outlooks, hail and wind forecasts exhibit an overforecast bias, while tornado and Day 2 and 3 outlooks exhibit a slight underforecast bias.…”

Global Ensemble Forecast System (GEFS) Predictions of Days 1–15 U.S. Tornado and Hail Frequencies

“…More recently, storm‐scale outputs from convection permitting models provide short‐range (e.g., 12–36 hr) forecast guidance (Gallo et al, , ). Accurately forecasting severe weather events remains a challenging task, but skill of SPC days 1–3 forecasts for severe convective storms has been increasing since the mid‐1990s (Herman et al, ; Hitchens & Brooks, ; ; Hitchens et al, ).…”

The Extended-Range Tornado Activity Forecast (ERTAF) Project

“…From an operational perspective, early modern forecasts for SCS events were confined to the same or next day (Grice et al 1999), but it is now commonplace to see official forecast outlooks for SCSs-issued by the NOAA/NWS's Storm Prediction Center (SPC)-with leads of 4-8 days. SPC forecasts have shown increasing skill with respect to time since the mid-1990s (Hitchens andBrooks 2012, 2014;Herman et al 2018); however, consistent skillful forecasts of daily tornado and hail activity are not likely to be found beyond day 9 and 12, respectively, if solely using current atmospheric numerical weather prediction systems (Gensini and Tippett 2019). Despite the limitations of dynamical prediction, other subseasonal forecasting techniques can act as a "bridge" to extend prediction beyond the traditional limits of NWP (Zhang 2013).…”