Bluecat: A Local Uncertainty Estimator for Deterministic Simulations and Predictions

Abstract

<p>We present a new method for simulating and predicting hydrologic variables with uncertainty assessment and provide example applications to river flows. The method is identified with the acronym ``Bluecat'' and is based on the use of a deterministic model which is subsequently converted to a stochastic formulation. The latter provides an adjustment on statistical basis of the deterministic prediction along with its confidence limits. The distinguishing features of the proposed approach are the ability to infer the probability distribution of the prediction without requiring strong hypotheses on the statistical characterization of the prediction error (e.g. normality, homoscedasticity) and its transparent and intuitive use of the observations. Bluecat makes use of a rigorous theory to estimate the probability distribution of the predictand conditioned by the deterministic model output, by inferring the conditional statistics of observations. Therefore Bluecat bridges the gaps between deterministic (possibly physically-based, or deep learning-based) and stochastic models as well as between rigorous theory and transparent use of data with an innovative and user oriented approach. We present two examples of application to the case studies of the Arno river at Subbiano and Sieve river at Fornacina. The results confirm the distinguishing features of the method along with its technical soundness. We provide an open software working in the R environment, along with help facilities and detailed instructions to reproduce the case studies presented here.</p>

A Two‐Stage Framework for Bias and Reliability Tests of Ensemble Hydroclimatic Forecasts

“…Nevertheless, the two test statistics are demonstrated to be robust tools for the examinations of bias and reliability. Their effectiveness is attributable to the fact that the PIT calculation employs the CDF of ensemble forecasts to normalize the observation and therefore mitigates the influence of extreme values (C. Jiang et al., 2015; Koutsoyiannis & Montanari, 2022; Q. J. Wang et al., 2020).…”

“…The PIT uniform plots presented in Figure 1 provide informative visualization of bias and reliability for ensemble hydroclimatic forecasts (J. Xu et al., 2022; Koutsoyiannis & Montanari, 2022; Q. J. Wang et al., 2020; Shao et al., 2021; Zhao et al., 2017). In this paper, the two‐stage framework along with two test statistics are developed to explicitly identify the five types of ensemble forecasts categorized by the PIT uniform plots.…”

“…The PIT uniform plot is one of the most popular diagnostic tools to examine the bias and reliability of ensemble hydroclimatic forecasts (Bennett et al., 2016; Koutsoyiannis & Montanari, 2022; Q. J. Wang et al., 2020; Schepen et al., 2018; Shao et al., 2021). Specifically, the PIT uniform plot takes advantage of the multiple members of ensemble forecasts: $$$\begin{array}{cccc}{F}_{k}=\left[\begin{array}{cccccccccccccccc}{f}_{k,1}& {f}_{k,2}& ...& {f}_{k,N}\end{array}\right]& (k=1,2,...,K)\end{array}$$$ in which $${F}_{k}$$ is the k th ensemble forecasts and $${f}_{k,n}$$ $$(n=1,2,...,N)$$ is the n th member of $${F}_{k}$$.…”

“…The use of the PIT uniform plots to empirically showcase the bias and reliability of ensemble hydroclimatic forecasts is presented in Figure 1. Conceptually, the five different shapes of PIT uniform plots correspond to overall five types of ensemble forecasts (Koutsoyiannis & Montanari, 2022; Q. J. Wang et al., 2009; Renard et al., 2010):…”

“…The quantile range plot shows ensemble forecasts and the corresponding observations against the ensemble median and it pays attention to the correspondence of high or low values of forecasts to high or low values of observations (Huang et al., 2021; Q. J. Wang et al., 2009; Zhao et al., 2019). Moreover, the probability integral transform (PIT) uniform plot illustrates the capability of ensemble spread in capturing the distribution of observations by empirical visualization of the distribution of PIT values (J. Xu et al., 2022; Koutsoyiannis & Montanari, 2022; Shao et al., 2021; Q. J. Wang et al., 2020; Zhao et al., 2017).…”