The role of statisticians in international science policy

Guttorp, Peter

doi:10.1002/env.1109

Cited by 9 publications

(4 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…More than a decade ago, Guttorp (2011) formulated a vision of how climate models should be evaluated against data (P. 820), “ …Climate models are di?cult to compare to data. Often climatologists compute some summary statistic, such as global annual mean temperature, and compare climate models using observed (or rather estimated) forcings to the observed (or rather estimated) temperatures.…”

Section: Introduction and Scopementioning

confidence: 99%

Distribution‐Based Model Evaluation and Diagnostics: Elicitability, Propriety, and Scoring Rules for Hydrograph Functionals

Vrugt

2024

Water Resources Research

View full text Add to dashboard Cite

Distribution forecasts P over future quantities or events are routinely made in hydrology but usually traded for a (likelihood‐weighted) mean or median prediction to accommodate error measures or scoring functions such as the mean absolute error or mean squared error. Case in point is the so‐called KG efficiency (KGE) of Gupta et al. (2009, https://doi.org/10.1016/j.jhydrol.2009.08.003) and improvements thereof (Lamontagne et al., 2020, https://doi.org/10.1029/2020wr027101), which have rapidly gained popularity among hydrologists as alternative scoring functions to the commonly used Nash and Sutcliffe (1970, https://doi.org/10.1016/0022‐1694(70)90255‐6) efficiency, but are equally exclusive in how they quantify model performance using only single‐valued output of the quantities of interest. This point‐valued mapping necessarily implies a loss of information about model performance. This paper advocates the use of probabilistic watershed model training, evaluation and diagnostics. Distribution evaluation opens a mature literature on scoring rules whose strong statistical underpinning provides, as we will demonstrate, the theory, context and guidelines necessary for the development of robust information‐theoretically principled metrics for watershed signatures. These so‐called hydrograph functionals are scalar‐valued mappings of major behavioral watershed functions embodied in a strictly proper scoring rule. We discuss past developments that led to the current state‐of‐the‐art of distribution evaluation in hydrology and review scoring rules for dichotomous and categorical events, quantiles (intervals) and density forecasts. We are particularly concerned with elicitable functionals and scoring rule propriety, discuss the decomposition of scoring rules into a sharpness, reliability and entropy term and present diagnostically appealing strictly proper divergence scores of hydrograph functionals for flood frequency analysis, flow duration and recession curves. The usefulness and power of distribution‐based model evaluation and diagnostics by means of scoring rules is demonstrated on simple illustrative problems and discharge distributions simulated with watershed models using random sampling and Bayesian model averaging. The presented theory (a) enables a more complete evaluation of distribution forecasts, (b) offers a statistically principled means for watershed model training, evaluation, diagnostics and selection using hydrograph functionals and/or extreme events and (c) provides a universal framework for metric development of watershed signatures, promoting metric standardization and reproducibility.

show abstract

Section: Introduction and Scopementioning

confidence: 99%

Distribution‐Based Model Evaluation and Diagnostics: Elicitability, Propriety, and Scoring Rules for Hydrograph Functionals

Vrugt

2024

Water Resources Research

View full text Add to dashboard Cite

show abstract

“…There is an important discussion in society (e.g., Guttorp, ; Howe, Markowitz, Lee, Ko, & Leiserowitz, ; Hulme, Obermeister, Randalls, & Borie, ; Karl & Trenberth, ; Kerr, ; King, ; Robinson & Shine, ) about the patterns of climate change observed in recent years, whether these are caused by human actions (Kaufmann, Kauppi, Mann, & Stock, ) or are the result of natural trends, and the impact of these changes on the environment and economy. A key issue in this problem is the correct measurement of climate change patterns and especially if there are changes in trends and cyclical patterns of climatic measures such as temperature and rainfall.…”

Section: Introductionmentioning

confidence: 99%

A spatio‐temporal approach to estimate patterns of climate change

Laurini

2018

Environmetrics

View full text Add to dashboard Cite

We introduce a method for decomposition of trend, cycle, and seasonal components in spatio‐temporal models and apply it to investigate the existence of climate changes in temperature series. The method incorporates critical features in the analysis of climatic problems—the importance of spatial heterogeneity, information from a large number of weather stations, and the presence of missing data. The spatial component is based on continuous projections of spatial covariance functions, allowing the modeling of complex patterns of dependence observed in climatic data. We apply this method to study climate changes in the northeast region of Brazil, characterized by a great wealth of climates and large amplitudes of temperatures. The results show the presence of a tendency for temperature increases, indicating changes in the climatic patterns in this region.

show abstract

“…In other words, a proper scoring rule is such that the expected score or penalty is minimized when the predictive distribution, F , agrees with the true distribution, G, of the quantity or event to be forecast. As the use of proper scoring rules encourages honest and careful assessments, 3 and is deeply entrenched in first decision theoretic principles, it is considered essential in scientific and managerial practice [11,12].…”

Section: Introductionmentioning

confidence: 99%

Using Proper Divergence Functions to Evaluate Climate Models

Thorarinsdottir¹,

Gneiting²,

Gissibl³

2013

SIAM/ASA J. Uncertainty Quantification

View full text Add to dashboard Cite

Abstract. It has been argued persuasively that, in order to evaluate climate models, the probability distributions of model output need to be compared to the corresponding empirical distributions of observed data. Distance measures between probability distributions, also called divergence functions, can be used for this purpose. We contend that divergence functions ought to be proper, in the sense that acting on modelers' true beliefs is an optimal strategy. The score divergences introduced in this paper derive from proper scoring rules and, thus, they are proper with the integrated quadratic distance and the Kullback-Leibler divergence being particularly attractive choices. Other commonly used divergences fail to be proper. In an illustration, we evaluate and rank simulations from 15 climate models for temperature extremes in a comparison to reanalysis data. 1. Introduction. Traditionally, climate models have been assessed by comparing summary statistics or point estimates that derive from the simulated model output to the corresponding observed quantities [1,2]. However, there is a growing consensus that in order to evaluate climate models the probability distribution of model output needs to be compared to the corresponding distribution of observed data. Guttorp [3, p. 820] argues powerfully the following:Climate models are difficult to compare to data. Often climatologists compute some summary statistic, such as global annual mean temperature, and compare climate models using observed (or rather estimated) forcings to the observed (or rather estimated) temperatures. However, it seems more appropriate to compare the distribution (over time and space) of climate model output to the corresponding distribution of observed data, as opposed to point estimates with or without confidence intervals.

show abstract

The role of statisticians in international science policy

Cited by 9 publications

References 17 publications

Distribution‐Based Model Evaluation and Diagnostics: Elicitability, Propriety, and Scoring Rules for Hydrograph Functionals

Distribution‐Based Model Evaluation and Diagnostics: Elicitability, Propriety, and Scoring Rules for Hydrograph Functionals

A spatio‐temporal approach to estimate patterns of climate change

Using Proper Divergence Functions to Evaluate Climate Models

Contact Info

Product

Resources

About