Application of Spatial Verification Methods to Idealized and NWP-Gridded Precipitation Forecasts

Ahijevych, David; Gilleland, Eric; Brown, Barbara G.; Ebert, Elizabeth E.

doi:10.1175/2009waf2222298.1

Cited by 98 publications

(108 citation statements)

References 19 publications

Supporting

Mentioning

101

Contrasting

Unclassified

Order By: Relevance

“…In addition to the traditional statistical scores, precipitation forecasts are verified by spatial verification methods, which not only consider the exact match of forecast and verification values at individual points but also take into account the matching of forecasts and observations in terms of objects or spatial scales (Casati et al, 2008;Ahijevych et al, 2009;Gilleland et al, 2010). This is necessary as precipitation fields exhibit high spatial variability and discontinuity.…”

Section: T Schellander-gorgas Et Al: On the Forecast Skill Of A Conmentioning

confidence: 99%

On the forecast skill of a convection-permitting ensemble

Schellander-Gorgas¹,

Wang²,

Meier³

et al. 2017

Geosci. Model Dev.

View full text Add to dashboard Cite

Abstract. The 2.5 km convection-permitting (CP) ensemble AROME-EPS (Applications of Research to Operations at Mesoscale -Ensemble Prediction System) is evaluated by comparison with the regional 11 km ensemble ALADIN-LAEF (Aire Limitée Adaption dynamique Développement InterNational -Limited Area Ensemble Forecasting) to show whether a benefit is provided by a CP EPS. The evaluation focuses on the abilities of the ensembles to quantitatively predict precipitation during a 3-month convective summer period over areas consisting of mountains and lowlands. The statistical verification uses surface observations and 1 km × 1 km precipitation analyses, and the verification scores involve state-of-the-art statistical measures for deterministic and probabilistic forecasts as well as novel spatial verification methods. The results show that the convectionpermitting ensemble with higher-resolution AROME-EPS outperforms its mesoscale counterpart ALADIN-LAEF for precipitation forecasts. The positive impact is larger for the mountainous areas than for the lowlands. In particular, the diurnal precipitation cycle is improved in AROME-EPS, which leads to a significant improvement of scores at the concerned times of day (up to approximately one-third of the scored verification measure). Moreover, there are advantages for higher precipitation thresholds at small spatial scales, which are due to the improved simulation of the spatial structure of precipitation.

show abstract

Section: T Schellander-gorgas Et Al: On the Forecast Skill Of A Conmentioning

confidence: 99%

On the forecast skill of a convection-permitting ensemble

Schellander-Gorgas¹,

Wang²,

Meier³

et al. 2017

Geosci. Model Dev.

View full text Add to dashboard Cite

show abstract

“…Building from the first intercomparison. The major results of the first spatial verification methods intercomparison project are summarized in Gilleland et al (2009Gilleland et al ( , 2010a and Ahijevych et al (2009). Through that project, the spatial verification approaches were categorized into four classes:…”

Section: September 2018mentioning

confidence: 99%

“…The project focused on prescribed errors for idealized cases and quantitative precipitation forecasts, where the ability of the verification methods to diagnose the known error was assessed (Ahijevych et al 2009). For this first intercomparison, complex terrain was considered too problematic, and therefore a test dataset from a region with relatively flat terrain was chosen, namely, the Great Plains of the United States.…”

Section: The Setup Of the Mesovict Projectmentioning

confidence: 99%

“…Objective results of the different spatial verification methods were therefore compared to a subjective assessment (Ahijevych et al 2009). The experiences gained during this process revealed just how challenging subjective assessment can be, as there was considerable disagreement between the assessors, depending on how each ranked the forecast attributes from the most to the least important.…”

Section: Experiments Designmentioning

confidence: 99%

See 1 more Smart Citation

The Setup of the MesoVICT Project

Dorninger

Gilleland

Casati

et al. 2018

Bulletin of the American Meteorological Society

Self Cite

View full text Add to dashboard Cite

MesoVICT focuses on the application, capability, and enhancement of spatial verification methods as applied to deterministic and ensemble forecasts of precipitation, wind, and temperature over complex terrain and includes observation uncertainty assessment. THE SETUP OF THE MesoVICT PROJECTManfreD Dorninger, eriC gillelanD, BarBara Casati, Marion P. MitterMaier, elizaBeth e. eBert, BarBara g. Brown, anD laurenCe J. wilson A s numerical weather prediction (NWP) models began to increase considerably in resolution, it became clear that traditional gridpoint-bygridpoint verification methods did not provide adequate or sufficient diagnostic information about forecast performance for some users (e.g., Mass et al. 2002). Double penalties arising from spatial displacement (or perhaps timing) errors and more rapid growth of small-scale errors often result in poorer performance scores for higher-resolution forecasts than their coarser counterparts, even when subjective evaluation would support the higher-resolution models as being better. Subsequently, a host of new verification methods were developed very rapidly (Ebert and McBride 2000;Harris et al. 2001;Casati et al. 2004;Nachamkin 2004;Davis et al. 2006;Keil and Craig 2007;Roberts and Lean 2008;Marzban et al. 2009;Gilleland et al. 2010b), which we will refer to as spatial methods. There still are gaps in our understanding when it comes to interpreting what all the new spatial methods tell us; gaining an in-depth understanding of forecast performance depends on grasping the full meaning of the verification results. Furthermore, the investment required to implement a new spatial method is relatively high compared to traditional verification methods, making it important to have some criteria on which to decide which methods would best suit a particular user's need(s). Therefore, a spatial methods meta-verification, or intercomparison, project was created to try to answer some of these questions.The first spatial verification methods intercomparison project (ICP;Gilleland et al. 2009Gilleland et al. , 2010a was initiated in 2007 with the aim of better understanding the rapidly increasing literature concerning new spatial verification methods, and several questions were addressed:• How does each method inform about forecast performance overall?• Which aspects of forecast error does each method best identify (e.g., location error, scale dependence of the skill, etc.)? • Which methods yield identical information to each other and which methods provide complementary information?The aim of the ICP was to analyze the behavior and provide a structured and systematic cataloging of the existing spatial verification methods, with the final goal of providing guidance to the users on which methods are best for which purpose. The project focused on prescribed errors for idealized cases and quantitative precipitation forecasts, where the ability of the verification methods to diagnose the known error was assessed (Ahijevych et al. 2009). For this first intercomparison, complex terrain ...

show abstract

“…These particular features were selected after an extensive evaluation of possible alternatives [texture based, geometric, and morphological (Lee et al 1985;Kumar and Foufoula-Georgiou 1994)]. These features are similar to the ones used in verification of spatial precipitation forecasts Ahijevych et al 2009;Gilleland et al 2010;Wolff et al 2014). Orientation is the angle between the x axis and the major axis of the ellipse that has the same second moments as the precipitation area in the image.…”

Section: Basis Vector Selectionmentioning

confidence: 99%

Quantifying Precipitation Uncertainty for Land Data Assimilation Applications

Alemohammad

McLaughlin

Entekhabi

2015

Monthly Weather Review

View full text Add to dashboard Cite

Ensemble-based data assimilation techniques are often applied to land surface models in order to estimate components of terrestrial water and energy balance. Precipitation forcing uncertainty is the principal source of spread among the ensembles that is required for utilizing information in observations to correct model priors. Precipitation fields may have both position and magnitude errors. However, current uncertainty characterizations of precipitation forcing in land data assimilation systems often do no more than applying multiplicative errors to precipitation fields. In this paper, an ensemble-based Bayesian method for characterization of uncertainties associated with precipitation retrievals from spaceborne instruments is introduced. This method is used to produce stochastic replicates of precipitation fields that are conditioned on precipitation observations. Unlike previous studies, the error likelihood is derived using an archive of historical measurements. The ensemble replicates are generated using a stochastic method, and they are intermittent in space and time. The replicates are first projected in a low-dimension subspace using a problem-specific set of attributes. The attributes are derived using a dimensionality reduction scheme that takes advantage of singular value decomposition. A nonparametric importance sampling technique is formulated in terms of the attribute vectors to solve the Bayesian sampling problem. Examples are presented using retrievals from operational passive microwave instruments, and performance of the method is assessed using ground validation measurements from a surface weather radar network. Results indicate that this ensemble characterization approach provides a useful description of precipitation uncertainties with a posterior ensemble that is narrower in distribution than its prior while containing both precipitation position and magnitude errors.

show abstract

Application of Spatial Verification Methods to Idealized and NWP-Gridded Precipitation Forecasts

Cited by 98 publications

References 19 publications

On the forecast skill of a convection-permitting ensemble

On the forecast skill of a convection-permitting ensemble

The Setup of the MesoVICT Project

Quantifying Precipitation Uncertainty for Land Data Assimilation Applications

Contact Info

Product

Resources

About