Spatial Forecast Verification Methods Intercomparison Project: Application of the SAL Technique

Wernli, Heini; Hofmann, Christiane; Zimmer, Matthias

doi:10.1175/2009waf2222271.1

Cited by 71 publications

(79 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The ICP was highly informative not only for end users, but also for the verification method developers, and it led to several improvements in the verification approaches (e.g., Keil and Craig 2009 Marzban et al 2009;Nachamkin 2009;Wernli et al 2009;Casati 2010;Gilleland et al 2010b; Lack et al 2010;Lakshmanan and Kain 2010;Mittermaier and Roberts 2010).…”

Section: The Setup Of the Mesovict Projectmentioning

confidence: 99%

“…Results for specific methods from the first intercomparison of spatial verification methods can be found in the Weather and Forecasting ICP special collection (https://journals.ametsoc.org/topic/verification_icp; Brill and Mesinger 2009;Davis et al 2009;Ebert and Gallus 2009;Keil and Craig 2009;Marzban et al 2009;Nachamkin 2009;Wernli et al 2009;Casati 2010;Gilleland et al 2010b;Lack et al 2010;Lakshmanan and Kain 2010;Mittermaier and Roberts 2010).…”

Section: The Setup Of the Mesovict Projectmentioning

confidence: 99%

“…Moreover, the verification methods were tested on nine selected case studies from the 2005 National Severe Storms Laboratory (NSSL) and Storm Prediction Center (SPC) Spring Experiment, using Stage II precipitation analyses as the verification fields, and the results were compared to human subjective assessments. The focus then was on the existing NWP operational capability: convection-permitting 4-7-km grids and the challenges these grid spacings presented for verifying precipitation forecasts.The ICP was highly informative not only for end users, but also for the verification method developers, and it led to several improvements in the verification approaches (e.g., Keil and Craig 2009 Marzban et al 2009;Nachamkin 2009;Wernli et al 2009;Casati 2010;Gilleland et al 2010b; Lack et al 2010;Lakshmanan and Kain 2010;Mittermaier and Roberts 2010).The ICP helped to promote the mainstream adoption of spatial verification approaches. Since the ICP special collection, a large number of papers have been published making use of the newer spatial techniques.…”

mentioning

confidence: 99%

See 2 more Smart Citations

The Setup of the MesoVICT Project

Dorninger

Gilleland

Casati

et al. 2018

Bulletin of the American Meteorological Society

View full text Add to dashboard Cite

MesoVICT focuses on the application, capability, and enhancement of spatial verification methods as applied to deterministic and ensemble forecasts of precipitation, wind, and temperature over complex terrain and includes observation uncertainty assessment. THE SETUP OF THE MesoVICT PROJECTManfreD Dorninger, eriC gillelanD, BarBara Casati, Marion P. MitterMaier, elizaBeth e. eBert, BarBara g. Brown, anD laurenCe J. wilson A s numerical weather prediction (NWP) models began to increase considerably in resolution, it became clear that traditional gridpoint-bygridpoint verification methods did not provide adequate or sufficient diagnostic information about forecast performance for some users (e.g., Mass et al. 2002). Double penalties arising from spatial displacement (or perhaps timing) errors and more rapid growth of small-scale errors often result in poorer performance scores for higher-resolution forecasts than their coarser counterparts, even when subjective evaluation would support the higher-resolution models as being better. Subsequently, a host of new verification methods were developed very rapidly (Ebert and McBride 2000;Harris et al. 2001;Casati et al. 2004;Nachamkin 2004;Davis et al. 2006;Keil and Craig 2007;Roberts and Lean 2008;Marzban et al. 2009;Gilleland et al. 2010b), which we will refer to as spatial methods. There still are gaps in our understanding when it comes to interpreting what all the new spatial methods tell us; gaining an in-depth understanding of forecast performance depends on grasping the full meaning of the verification results. Furthermore, the investment required to implement a new spatial method is relatively high compared to traditional verification methods, making it important to have some criteria on which to decide which methods would best suit a particular user's need(s). Therefore, a spatial methods meta-verification, or intercomparison, project was created to try to answer some of these questions.The first spatial verification methods intercomparison project (ICP;Gilleland et al. 2009Gilleland et al. , 2010a was initiated in 2007 with the aim of better understanding the rapidly increasing literature concerning new spatial verification methods, and several questions were addressed:• How does each method inform about forecast performance overall?• Which aspects of forecast error does each method best identify (e.g., location error, scale dependence of the skill, etc.)? • Which methods yield identical information to each other and which methods provide complementary information?The aim of the ICP was to analyze the behavior and provide a structured and systematic cataloging of the existing spatial verification methods, with the final goal of providing guidance to the users on which methods are best for which purpose. The project focused on prescribed errors for idealized cases and quantitative precipitation forecasts, where the ability of the verification methods to diagnose the known error was assessed (Ahijevych et al. 2009). For this first intercomparison, complex terrain ...

show abstract

Section: The Setup Of the Mesovict Projectmentioning

confidence: 99%

Section: The Setup Of the Mesovict Projectmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

The Setup of the MesoVICT Project

Dorninger

Gilleland

Casati

et al. 2018

Bulletin of the American Meteorological Society

View full text Add to dashboard Cite

show abstract

“…This choice is made to assess the impact of the assimilation of radar reflectivity volumes in the region where these data are actually observed. Furthermore, regarding SAL, in Wernli et al (2009) it is recommended to use a domain not larger than 500 × 500 km 2 since, otherwise, the domain may include different meteorological systems making the interpretation of results …”

Section: Verificationmentioning

confidence: 99%

Data assimilation of radar reflectivity volumes in a LETKF scheme

Gastaldo¹,

Poli²,

Marsigli³

et al. 2018

Preprint

View full text Add to dashboard Cite

Abstract. Quantitative precipitation forecast (QPF) is still a challenge for numerical weather prediction (NWP), despite the continuous improvement of models and data assimilation systems. In this regard, the assimilation of radar reflectivity volumes should be beneficial, since the accuracy of analysis is the element that most affects short-term QPFs. Up to now, very few attempts have been made to assimilate these observations in an operational set-up, due to the large amount of computational resources needed and to several open issues, like the arise of imbalances in the analyses and the estimation of the observational 5

show abstract

“…The primary purpose of this study is to use the displacement-type method SAL (which stands for structure, amplitude, and location) developed by Wernli et al (2008), to evaluate precipitation forecasts during the convective season over a wide range of model resolutions against a high-resolution precipCorrespondence to: C. Wittmann (christoph.wittmann@zamg.ac.at) itation analysis. Previously, raingauge data interpolated to a 5 km grid (Früh et al, 2007) and a 10 km grid (Wernli et al, 2008), or radar rainfall data were used, the latter however only for a 9-day period (Wernli et al, 2009). Also, we compare NWP models over a broader range of resolutions than has been done in the past, with grid spacings ranging from 2.5 to 25 km.…”

Section: Introductionmentioning

confidence: 99%

Evaluating multi-scale precipitation forecasts using high resolution analysis

2010

View full text Add to dashboard Cite

Abstract. The SAL (Structure, Amplitude, Location) method is used for verification of precipitation forecasts at horizontal grid spacings ranging from 2.5 km to 25 km, using a high-resolution 1 km precipitation analysis as a reference. The verification focuses on a summertime period with predominantly convective precipitation. The verification domain contains lowland as well as alpine areas. Evaluation of the individual SAL components shows that with regard to area mean values (A) the benefit of high resolutions models becomes apparent only in high impact weather situations. For the summertime period studied, the subjective impression of better structured precipitation fields (S ) in higher resolution models can generally be confirmed. The most significant improvement appears to be associated with explicit simulation of deep convection.

show abstract

Spatial Forecast Verification Methods Intercomparison Project: Application of the SAL Technique

Cited by 71 publications

References 12 publications

The Setup of the MesoVICT Project

The Setup of the MesoVICT Project

Data assimilation of radar reflectivity volumes in a LETKF scheme

Evaluating multi-scale precipitation forecasts using high resolution analysis

Contact Info

Product

Resources

About