Using Proper Divergence Functions to Evaluate Climate Models

Thorarinsdottir, Thordis L.; Gneiting, Tilmann; Gissibl, Nadine

doi:10.1137/130907550

Cited by 66 publications

(66 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, Candille and Talagrand (2008) use the quadratic divergence, ( f − g ) 2 , in the case of forecasting a binary event (also Santos and Ghelli, 2012), Pappenberger et al (2009) use the Kullback–Leibler divergence (or relative entropy),

\int g normallog false(g false/ f false)

, and Friederichs and Thorarinsdottir (2012) propose the integrated quadratic distance,

\int {false(f - g false)}^{2}

. Thorarinsdottir et al (2013) list several other divergences, including the sub‐class of ‘score divergences’ that are formed from proper scoring rules, s , in the following way:

d (f, g) = {normalE}_{y} {s (f, y)} - {normalE}_{y} {s (g, y)},

where y ∼ g . Score divergences thus differ from expected (proper) scoring rules by subtracting an amount, E y { s ( g , y )}, which is independent of the forecast.…”

Section: Other Approaches To Observation Errormentioning

confidence: 99%

“…Other authors measure the difference between f and g with a divergence, that is a function, d(f , g), for which d(g, g) = 0 and d(f , g) ≥ 0 for all f and g. For example, Candille and Talagrand (2008) use the quadratic divergence, (f − g) 2 , in the case of forecasting a binary event (also Santos and Ghelli, 2012), Pappenberger et al (2009) use the Kullback-Leibler divergence (or relative entropy), g log(g/f ), and Friederichs and Thorarinsdottir (2012) propose the integrated quadratic distance, (f − g) 2 . Thorarinsdottir et al (2013) list several other divergences, including the sub-class of 'score divergences' that are formed from proper scoring rules, s, in the following way:…”

Section: Probabilistic Observationsmentioning

confidence: 99%

See 1 more Smart Citation

Measuring forecast performance in the presence of observation error

Ferro

2017

Quart J Royal Meteoro Soc

View full text Add to dashboard Cite

A new framework is introduced for measuring the performance of probability forecasts when the true value of the predictand is observed with error. In these circumstances, proper scoring rules favour good forecasts of observations rather than of truth and yield scores that vary with the quality of the observations. Proper scoring rules thus can favour forecasters who issue worse forecasts of the truth and can mask real changes in forecast performance if observation quality varies over time. Existing approaches to accounting for observation error provide unsatisfactory solutions to these two problems. A new class of ‘error‐corrected’ proper scoring rules is defined that solves both problems by producing unbiased estimates of the scores that would be obtained if the forecasts could be verified against the truth. A general method for constructing error‐corrected proper scoring rules is given for the case of categorical predictands, and error‐corrected versions of the Dawid–Sebastiani scoring rule are proposed for numerical predictands. The benefits of accounting for observation error in ensemble post‐processing and in forecast verification are illustrated in three data examples that include forecasts for the occurrence of tornadoes and of aircraft icing.

show abstract

\int g normallog false(g false/ f false)

, and Friederichs and Thorarinsdottir (2012) propose the integrated quadratic distance,

\int {false(f - g false)}^{2}

. Thorarinsdottir et al (2013) list several other divergences, including the sub‐class of ‘score divergences’ that are formed from proper scoring rules, s , in the following way:

d (f, g) = {normalE}_{y} {s (f, y)} - {normalE}_{y} {s (g, y)},

where y ∼ g . Score divergences thus differ from expected (proper) scoring rules by subtracting an amount, E y { s ( g , y )}, which is independent of the forecast.…”

Section: Other Approaches To Observation Errormentioning

confidence: 99%

Section: Probabilistic Observationsmentioning

confidence: 99%

Measuring forecast performance in the presence of observation error

Ferro

2017

Quart J Royal Meteoro Soc

View full text Add to dashboard Cite

show abstract

“…The divergence d ( P,Q ) = S ( P,Q ) − S ( Q,Q ) is the non‐negative difference between the expected score and the entropy. Note that propriety of the score function implies propriety of the divergence (Thorarinsdottir et al 2014). d ( P,Q ) represents a measure for similarity between the probabilistic forecast P and the distribution Q , where smaller values indicate better correspondence.…”

Section: Proper Score Functionsmentioning

confidence: 99%

Decomposition and graphical portrayal of the quantile score

Bentzien

Friederichs

2014

Q.J.R. Meteorol. Soc.

View full text Add to dashboard Cite

This study expands the pool of verification methods for probabilistic weather and climate predictions by a decomposition of the quantile score (QS). The QS is a proper score function and evaluates predictive quantiles on a set of forecast-observation pairs. We introduce a decomposition of the QS in reliability, resolution and uncertainty and discuss the biases of the decomposition. Further, a reliability diagram for quantile forecasts is presented. Verification with the QS and its decomposition is illustrated on precipitation forecasts derived from the mesoscale weather prediction ensemble COSMO-DE-EPS of the German Meteorological Service. We argue that the QS is ready to become as popular as the Brier score in forecast verification.

show abstract

“…We compare two marginal distributions F and G using the integrated quadratic distance (IQD; Thorarinsdottir et al, 2013),…”

Section: Evaluation Methodsmentioning

confidence: 99%

New Approach for Bias Correction and Stochastic Downscaling of Future Projections for Daily Mean Temperatures to a High-Resolution Grid

Yuan

Thorarinsdottir

Beldring

et al. 2019

Journal of Applied Meteorology and Climatology

Self Cite

View full text Add to dashboard Cite

In applications of climate information, coarse-resolution climate projections commonly need to be downscaled to a finer grid. One challenge of this requirement is the modeling of sub-grid variability and the spatial and temporal dependence at the finer scale. Here, a post-processing procedure is proposed for temperature projections that addresses this challenge. The procedure employs statistical bias correction and stochastic downscaling in two steps. In a first step, errors that are related to spatial and temporal features of the first two moments of the temperature distribution at model scale are identified and corrected. Secondly, residual spacetime dependence at the finer scale is analyzed using a statistical model, from which realizations are generated and then combined with appropriate climate change signal to form the downscaled projection fields. Using a high-resolution observational gridded data product, the proposed approach is applied in a case study where projections of two regional climate models from the EURO-CORDEX ensemble are bias-corrected and downscaled to a 1 × 1 km grid in the Trøndelag area of Norway.A cross-validation study shows that the proposed procedure generates results that better reflect the marginal distributional properties of the data product and have better consistency in space and time than empirical quantile mapping.

show abstract

Using Proper Divergence Functions to Evaluate Climate Models

Cited by 66 publications

References 28 publications

Measuring forecast performance in the presence of observation error

Measuring forecast performance in the presence of observation error

Decomposition and graphical portrayal of the quantile score

New Approach for Bias Correction and Stochastic Downscaling of Future Projections for Daily Mean Temperatures to a High-Resolution Grid

Contact Info

Product

Resources

About