Application of Two Spatial Verification Methods to Ensemble Forecasts of Low-Level Rotation

Abstract

Two spatial verification methods are applied to ensemble forecasts of low-level rotation in supercells: a four-dimensional, object-based matching algorithm and the displacement and amplitude score (DAS) based on optical flow. Ensemble forecasts of low-level rotation produced using the National Severe Storms Laboratory (NSSL) Experimental Warn-on-Forecast System are verified against WSR-88D single-Doppler azimuthal wind shear values interpolated to the model grid. Verification techniques are demonstrated using four 60-min forecasts issued at 15-min intervals in the hour preceding development of the 20 May 2013 Moore, Oklahoma, tornado and compared to results from two additional forecasts of tornadic supercells occurring during the springs of 2013 and 2014. The object-based verification technique and displacement component of DAS are found to reproduce subjectively determined forecast characteristics in successive forecasts for the 20 May 2013 event, as well as to discriminate in subjective forecast quality between different events. Ensemble-mean, object-based measures quantify spatial and temporal displacement, as well as storm motion biases in predicted low-level rotation in a manner consistent with subjective interpretation. Neither method produces useful measures of the intensity of low-level rotation, owing to deficiencies in the verification dataset and forecast resolution.

Spatial Verification of Ensemble Precipitation: An Ensemble Version of SAL

“…Here, eS refers to the ability of the ensemble as a whole to simulate, on average, the right scaled volumes. The idea of using an ensemble average of feature characteristics is similar to the approaches used by Skinner et al (2016) or Gallus (2010). We tested an alternative definition of eS, taking the CRPS between distributions of scaled volumes, but this approach has the disadvantage of losing information on the sign, that is, if the simulated features are too small or too large, and also losing the equivalence with the standard SAL for purely deterministic cases.…”

“…Exceptions include the work of Gallus (2010), who averaged feature characteristics obtained with the contiguous rain area (CRA; Ebert and McBride 2000) and Method for Object-Based Diagnostic Evaluation (MODE; Davis et al 2006Davis et al , 2009 methods over all ensemble members containing a matched feature. A similar approach was proposed by Skinner et al (2016) for ensemble forecasts of mesocyclones. Further, Fox et al (2016) used Bayesian Procrustes shape analysis to define meaningful ensemble mean features, Schwartz et al (2017) compared locations of probability objects with locations of observed objects, and Mittermaier and Bullock (2013) and later Clark et al (2014) extended MODE to three-dimensional objects where the third dimension is the time dimension.…”

Application of the Warn‐on‐Forecast system for flash‐flood‐producing heavy convective rainfall events

“…The WoFS is a frequently cycled, convective‐scale, ensemble DA and prediction system. Multiple studies have shown the potential of the WoFS for 0–1‐hr probabilistic forecasts of severe thunderstorms and low‐level rotations (Yussouf et al, 2013a, 2013b; ; Wheatley et al ., ; Jones et al ., ; Skinner et al ., ). The system also demonstrated skillful rainfall forecasts of an intense convective rainfall and flash flood event (Yussouf et al ., ).…”

“…The Warn‐on‐Forecast (WoF) project (Stensrud et al ., , ) at the National Oceanic and Atmospheric Administration's (NOAA) National Severe Storms Laboratory (NSSL) aims to enable a new paradigm for National Weather Service (NWS) severe weather watch‐to‐warning operations, where convective‐scale probabilistic numerical weather prediction (NWP) model guidance is the key resource. The outcome of NSSL's several years of research and development efforts (Dawson et al ., ; Yussouf et al ., , , ; Wheatley et al ., ; ; Jones et al ., ; Skinner et al ., ) since the beginning of the WoF initiative in 2009 is an experimental Warn‐on‐Forecast System (WoFS). The system is a rapidly updating, regional, convective‐scale, on‐demand, ensemble data assimilation (DA) and prediction system that provides very short‐term probabilistic model guidance for tornadoes, heavy rainfall, large hail, damaging straight‐line winds and other convective hazards.…”

Verification of Near Surface Wind Patterns in Germany using Clear Air Radar Echoes

“…In addition, Doppler-derived wind composites are not yet widely available. Skinner et al (2016) present a very interesting alternative using single-Doppler azimuthal wind shear as a proxy for lowlevel rotation. Their study also highlights some of the main methodological challenges related to wind verification: Most spatial verification techniques were developed for scalar quantities which can be decomposed into discrete objects via thresholding.…”