Verification with Variograms

Abstract

The verification of a gridded forecast field, for example, one produced by numerical weather prediction (NWP) models, cannot be performed on a gridpoint-by-gridpoint basis; that type of approach would ignore the spatial structures present in both forecast and observation fields, leading to misinformative or noninformative verification results. A variety of methods have been proposed to acknowledge the spatial structure of the fields. Here, a method is examined that compares the two fields in terms of their variograms. Two types of variograms are examined: one examines correlation on different spatial scales and is a measure of texture; the other type of variogram is additionally sensitive to the size and location of objects in a field and can assess size and location errors. Using these variograms, the forecasts of three NWP model formulations are compared with observations/analysis, on a dataset consisting of 30 days in spring 2005. It is found that within statistical uncertainty the three formulations are comparable with one another in terms of forecasting the spatial structure of observed reflectivity fields. None, however, produce the observed structure across all scales, and all tend to overforecast the spatial extent and also forecast a smoother precipitation (reflectivity) field. A finer comparison suggests that the University of Oklahoma 2-km resolution Advanced Research Weather Research and Forecasting (WRF-ARW) model and the National Center for Atmospheric Research (NCAR) 4-km resolution WRF-ARW slightly outperform the 4.5-km WRF-Nonhydrostatic Mesoscale Model (NMM), developed by the National Oceanic and Atmospheric Administration/National Centers for Environmental Prediction (NOAA/NCEP), in terms of producing forecasts whose spatial structures are closer to that of the observed field.

Validation of the Meteorological Outputs of the Canadian Regional Climate Model Using a Kriging Method: Application to Southern Quebec

“…Nonetheless, the present study proposed a comprehensive kriging framework that could be used to compare different sources of data on the same spatial scale for different time steps. Let us mention that for a short time step, the proposed framework should be adapted to account for the possibility of zero precipitation (Marzban and Sandgathe, 2009). Second, only one CRCM run was used in this study.…”

“…In addition to being a necessary step before kriging interpolation, the variographic analysis is a verification method that illustrates the differences between observed and CRCM data caused by their different spatial resolutions. Other verification methods, like scale decomposition methods, could have been used, but variograms have the advantage to keep the data in their original field, allowing a better geographic understanding of the results (Marzban and Sandgathe, 2009 The exponential model was used for the annual and fall observed minimum temperature. The spherical model was used for observed winter data and a Gaussian model was used for observed spring and summer data.…”

“…To determine this function, an experimental semi-variogram must be constructed by calculating the mean half squared difference between pairs of sites as a function of their distance. An example of experimental semi-variogram construction is given by Marzban and Sandgathe (2009).…”

An Object-Based Approach for Quantification of GCM Biases of the Simulation of Orographic Precipitation. Part II: Quantitative Analysis

“…Clustering has been extensively used in meteorology and climatology (Huth et al 1993;Littmann 2000;Liu and George 2005). It has also been applied for object-based purposes including storm and cloud regime classification (Eitzen et al 2008;Jakob and Tselioudis 2003;Lakshmanan et al 2003;Peak and Tag 1994), verification of precipitation fields produced by weather prediction models (Marzban and Sandgathe 2006), and classification of multimodel ensemble data (Alhamed et al 2002). These studies employed hierarchical cluster analysis methods.…”

“…Section 3 gives information about both k-means cluster analysis and variography. Both methods have been used for evaluation of weather prediction and climate models (Alhamed et al 2002;Johnson et al 2011;Liu and George 2005;Marzban andSandgathe 2006, 2009). We apply them to idealized test case results (Yorgun and Rood 2014) with an object-based point of view.…”