Object-Based Verification of Short-Term, Storm-Scale Probabilistic Mesocyclone Guidance from an Experimental Warn-on-Forecast System

Abstract

An object-based verification method for short-term, storm-scale probabilistic forecasts was developed and applied to mesocyclone guidance produced by the experimental Warn-on-Forecast System (WoFS) in 63 cases from 2017 to 2018. The probabilistic mesocyclone guidance was generated by calculating gridscale ensemble probabilities from WoFS forecasts of updraft helicity (UH) in layers 2–5 km (midlevel) and 0–2 km (low-level) above ground level (AGL) aggregated over 60-min periods. The resulting ensemble probability swaths are associated with individual thunderstorms and treated as objects with a single, representative probability value prescribed. A mesocyclone probability object, conceptually, is a region bounded by the ensemble forecast envelope of a mesocyclone track for a given thunderstorm over 1 h. The mesocyclone probability objects were matched against rotation track objects in Multi-Radar Multi-Sensor data using the total interest score, but with the maximum displacement varied between 0, 9, 15, and 30 km. Forecast accuracy and reliability were assessed at four different forecast lead time periods: 0–60, 30–90, 60–120, and 90–150 min. In the 0–60-min forecast period, the low-level UH probabilistic forecasts had a POD, FAR, and CSI of 0.46, 0.45, and 0.31, respectively, with a probability threshold of 22.2% (the threshold of maximum CSI). In the 90–150-min forecast period, the POD and CSI dropped to 0.39 and 0.27 while FAR remained relatively unchanged. Forecast probabilities > 60% overpredicted the likelihood of observed mesocyclones in the 0–60-min period; however, reliability improved when allowing larger maximum displacements for object matching and at longer lead times.

Sensitivity of a Bowing Mesoscale Convective System to Horizontal Grid Spacing in a Convection-Allowing Ensemble

“…The single-nest ensemble does have more location error, by eye, in reflectivity fields; however, despite the low weight of the location component, this ensemble has more taSAL spread than the nested ensemble regardless (i.e., taSAL is a conservative estimate). Further work into bowing MCS evolution in EPSs should consider the probabilistic distribution of solutions by using an appropriate score that preserves the ensemble-output estimates of uncertainty (e.g., [40][41][42]).…”

Review article: Observations for high-impact weather and their use in verification

“…Evaluation from citizens used for verification. MeteoSwiss carried out a subjective verification by beta testers of thunderstorm warnings issued for municipalities on mobile phones via app (Gaia et al, 2017). The forecast was issued in categories: "a developing/moderate/severe/very severe thunderstorm is expected in the next XX min in a given municipality".…”

“…Therefore, the same verification approach can be applied in the context of ensemble forecasting (Marsigli et al, 2019). Verification of probability objects for thunderstorm forecast is performed in Flora et al (2019).…”

Observations for high-impact weather and their use in verification

“…Therefore, the same verification approach can be applied in the context of ensemble forecasting (Marsigli et al, 2019). Verification of probability objects for thunderstorm forecast is performed in Flora et al (2019).…”