An object-based verification methodology for the NSSL Experimental Warn-on-Forecast System for ensembles (NEWS-e) has been developed and applied to 32 cases between December 2015 and June 2017. NEWS-e forecast objects of composite reflectivity and 30-min updraft helicity swaths are matched to corresponding reflectivity and rotation track objects in Multi-Radar Multi-Sensor system data on space and time scales typical of a National Weather Service warning. Object matching allows contingency-table-based verification statistics to be used to establish baseline performance metrics for NEWS-e thunderstorm and mesocyclone forecasts. NEWS-e critical success index (CSI) scores of reflectivity (updraft helicity) forecasts decrease from approximately 0.7 (0.4) to 0.4 (0.2) over 3 h of forecast time. CSI scores decrease through the forecast period, indicating that errors do not saturate during the 3-h forecast. Lower verification scores for rotation track forecasts are primarily a result of a high-frequency bias. Comparison of different system configurations used in 2016 and 2017 shows an increase in skill for 2017 reflectivity forecasts, attributable mainly to improvements in the forecast initial conditions. A small decrease in skill in 2017 rotation track forecasts is likely a result of sample differences between 2016 and 2017. Although large case-to-case variation is present, evidence is found that NEWS-e forecast skill improves with increasing object size and intensity.

Two spatial verification methods are applied to ensemble forecasts of low-level rotation in supercells: a four-dimensional, object-based matching algorithm and the displacement and amplitude score (DAS) based on optical flow. Ensemble forecasts of low-level rotation produced using the National Severe Storms Laboratory (NSSL) Experimental Warn-on-Forecast System are verified against WSR-88D single-Doppler azimuthal wind shear values interpolated to the model grid. Verification techniques are demonstrated using four 60-min forecasts issued at 15-min intervals in the hour preceding development of the 20 May 2013 Moore, Oklahoma, tornado and compared to results from two additional forecasts of tornadic supercells occurring during the springs of 2013 and 2014. The object-based verification technique and displacement component of DAS are found to reproduce subjectively determined forecast characteristics in successive forecasts for the 20 May 2013 event, as well as to discriminate in subjective forecast quality between different events. Ensemble-mean, object-based measures quantify spatial and temporal displacement, as well as storm motion biases in predicted low-level rotation in a manner consistent with subjective interpretation. Neither method produces useful measures of the intensity of low-level rotation, owing to deficiencies in the verification dataset and forecast resolution.

An object-based verification method for short-term, storm-scale probabilistic forecasts was developed and applied to mesocyclone guidance produced by the experimental Warn-on-Forecast System (WoFS) in 63 cases from 2017 to 2018. The probabilistic mesocyclone guidance was generated by calculating gridscale ensemble probabilities from WoFS forecasts of updraft helicity (UH) in layers 2–5 km (midlevel) and 0–2 km (low-level) above ground level (AGL) aggregated over 60-min periods. The resulting ensemble probability swaths are associated with individual thunderstorms and treated as objects with a single, representative probability value prescribed. A mesocyclone probability object, conceptually, is a region bounded by the ensemble forecast envelope of a mesocyclone track for a given thunderstorm over 1 h. The mesocyclone probability objects were matched against rotation track objects in Multi-Radar Multi-Sensor data using the total interest score, but with the maximum displacement varied between 0, 9, 15, and 30 km. Forecast accuracy and reliability were assessed at four different forecast lead time periods: 0–60, 30–90, 60–120, and 90–150 min. In the 0–60-min forecast period, the low-level UH probabilistic forecasts had a POD, FAR, and CSI of 0.46, 0.45, and 0.31, respectively, with a probability threshold of 22.2% (the threshold of maximum CSI). In the 90–150-min forecast period, the POD and CSI dropped to 0.39 and 0.27 while FAR remained relatively unchanged. Forecast probabilities > 60% overpredicted the likelihood of observed mesocyclones in the 0–60-min period; however, reliability improved when allowing larger maximum displacements for object matching and at longer lead times.

Observations collected in the second Verification of the Origins of Rotation in Tornadoes Experiment during a 15-min period of a supercell occurring on 18 May 2010 near Dumas, Texas, are presented. The primary data collection platforms include two Ka-band mobile Doppler radars, which collected a near-surface, short-baseline dual-Doppler dataset within the rear-flank outflow of the Dumas supercell; an X-band, phasedarray mobile Doppler radar, which collected volumetric single-Doppler data with high temporal resolution; and in situ thermodynamic and wind observations of a six-probe mobile mesonet.Rapid evolution of the Dumas supercell was observed, including the development and decay of a low-level mesocyclone and four internal rear-flank downdraft (RFD) momentum surges. Intensification and upward growth of the low-level mesocyclone were observed during periods when the midlevel mesocyclone was minimally displaced from the low-level circulation, suggesting an upward-directed perturbation pressure gradient force aided in the intensification of low-level rotation. The final three internal RFD momentum surges evolved in a manner consistent with the expected behavior of a dynamically forced occlusion downdraft, developing at the periphery of the low-level mesocyclone during periods when values of low-level cyclonic azimuthal wind shear exceeded values higher aloft. Failure of the low-level mesocyclone to acquire significant vertical depth suggests that dynamic forcing above internal RFD momentum surge gust fronts was insufficient to lift the negatively buoyant air parcels comprising the RFD surges to significant heights. As a result, vertical acceleration and the stretching of vertical vorticity in surge parcels were limited, which likely contributed to tornadogenesis failure.

The increasing maturity of the Warn-on-Forecast System (WoFS) coupled with the now operational GOES-16 satellite allows for the first time a comprehensive analysis of the relative impacts of assimilating GOES-16 all-sky 6.2-, 6.9-, and 7.3-μm channel radiances compared to other radar and satellite observations. The WoFS relies on cloud property retrievals such as cloud water path, which have been proven to increase forecast skill compared to only assimilating radar data and other conventional observations. The impacts of assimilating clear-sky radiances have also been explored and shown to provide useful information on midtropospheric moisture content in the near-storm environment. Assimilation of all-sky radiances adds a layer of complexity and is tested to determine its effectiveness across four events occurring in the spring and summer of 2019. Qualitative and object-based verification of severe weather and the near-storm environment are used to assess the impact of assimilating all-sky radiances compared to the current model configuration. We focus our study through the entire WoFS analysis and forecasting cycle (1900–0600 UTC, daily) so that the impacts throughout the evolution of convection from initiation to large upscale growth can be assessed. Overall, assimilating satellite data improves forecasts relative to radar-only assimilation experiments. The retrieval method with clear-sky radiances performs best overall, but assimilating all-sky radiances does have very positive impacts in certain conditions. In particular, all-sky radiance assimilation improved convective initiation forecast of severe storms in several instances. This work represents an initial attempt at assimilating all-sky radiances into the WoFS and additional research is ongoing to further improve forecast skill.

A prototype convection-allowing system using the Advanced Research version of the Weather Research and Forecasting (WRF-ARW) Model and employing an ensemble Kalman filter (EnKF) data assimilation technique has been developed and used during the spring 2016 and 2017 Hazardous Weather Testbeds. This system assimilates WSR-88D reflectivity and radial velocity, geostationary satellite cloud water path (CWP) retrievals, and available surface observations over a regional domain with a 3-km horizontal resolution at 15-min intervals, with 3-km initial conditions provided by an experimental High-Resolution Rapid Refresh ensemble (HRRR-e). However, no information on upper-level thermodynamic conditions in cloud-free regions is currently assimilated, as few timely observations exist. One potential solution is to also assimilate clear-sky satellite radiances, which provide information on mid- and upper-tropospheric temperature and moisture conditions. This research assimilates GOES-13 imager water vapor band (6.5 μm) radiances using the GSI-EnKF system to take advantage of the Community Radiative Transfer Model (CRTM) integration. Results using four cases from May 2016 showed that assimilating radiances generally had a neutral-to-positive impact on the model analysis, reducing humidity bias and/or errors at the appropriate model levels where verification observations were present. The effects on high-impact weather forecasts, as verified against forecast reflectivity and updraft helicity, were mixed. Three cases (9, 22, and 24 May) showed some improvement in skill, while the other (25 May) performed worse, despite the improved environment. This research represents the first step in designing a high-resolution ensemble data assimilation system to use GOES-16 Advanced Baseline Imager data, which provides additional water vapor bands and increased spatial and temporal resolution.

The 2016–18 NOAA Hazardous Weather Testbed (HWT) Spring Forecasting Experiments (SFE) featured the Community Leveraged Unified Ensemble (CLUE), a coordinated convection-allowing model (CAM) ensemble framework designed to provide empirical guidance for development of operational CAM systems. The 2017 CLUE included 81 members that all used 3-km horizontal grid spacing over the CONUS, enabling direct comparison of forecasts generated using different dynamical cores, physics schemes, and initialization procedures. This study uses forecasts from several of the 2017 CLUE members and one operational model to evaluate and compare CAM representation and next-day prediction of thunderstorms. The analysis utilizes existing techniques and novel, object-based techniques that distill important information about modeled and observed storms from many cases. The National Severe Storms Laboratory Multi-Radar Multi-Sensor product suite is used to verify model forecasts and climatologies of observed variables. Unobserved model fields are also examined to further illuminate important intermodel differences in storms and near-storm environments. No single model performed better than the others in all respects. However, there were many systematic intermodel and intercore differences in specific forecast metrics and model fields. Some of these differences can be confidently attributed to particular differences in model design. Model intercomparison studies similar to the one presented here are important to better understand the impacts of model and ensemble configurations on storm forecasts and to help optimize future operational CAM systems.

The Storm Prediction Center (SPC) tornado database, generated from NCEI’s Storm Data publication, is indispensable for assessing U.S. tornado risk and investigating tornado–climate connections. Maximizing the value of this database, however, requires accounting for systemically lower reported tornado counts in rural areas owing to a lack of observers. This study uses Bayesian hierarchical modeling to estimate tornado reporting rates and expected tornado counts over the central United States during 1975–2016. Our method addresses a serious solution nonuniqueness issue that may have affected previous studies. The adopted model explains 73% (>90%) of the variance in reported counts at scales of 50 km (>100 km). Population density explains more of the variance in reported tornado counts than other examined geographical covariates, including distance from nearest city, terrain ruggedness index, and road density. The model estimates that approximately 45% of tornadoes within the analysis domain were reported. The estimated tornado reporting rate decreases sharply away from population centers; for example, while >90% of tornadoes that occur within 5 km of a city with population > 100 000 are reported, this rate decreases to <70% at distances of 20–25 km. The method is directly extendable to other events subject to underreporting (e.g., severe hail and wind) and could be used to improve climate studies and tornado and other hazard models for forecasters, planners, and insurance/reinsurance companies, as well as for the development and verification of storm-scale prediction systems.