Critical evaluation of parameter consistency and predictive uncertainty in hydrological modeling: A case study using Bayesian total error analysis

Abstract

[1] The lack of a robust framework for quantifying the parametric and predictive uncertainty of conceptual rainfall-runoff (CRR) models remains a key challenge in hydrology. The Bayesian total error analysis (BATEA) methodology provides a comprehensive framework to hypothesize, infer, and evaluate probability models describing input, output, and model structural error. This paper assesses the ability of BATEA and standard calibration approaches (standard least squares (SLS) and weighted least squares (WLS)) to address two key requirements of uncertainty assessment: (1) reliable quantification of predictive uncertainty and (2) reliable estimation of parameter uncertainty. The case study presents a challenging calibration of the lumped GR4J model to a catchment with ephemeral responses and large rainfall gradients. Postcalibration diagnostics, including checks of predictive distributions using quantile-quantile analysis, suggest that while still far from perfect, BATEA satisfied its assumed probability models better than SLS and WLS. In addition, WLS/SLS parameter estimates were highly dependent on the selected rain gauge and calibration period. This will obscure potential relationships between CRR parameters and catchment attributes and prevent the development of meaningful regional relationships. Conversely, BATEA provided consistent, albeit more uncertain, parameter estimates and thus overcomes one of the obstacles to parameter regionalization. However, significant departures from the calibration assumptions remained even in BATEA, e.g., systematic overestimation of predictive uncertainty, especially in validation. This is likely due to the inferred rainfall errors compensating for simplified treatment of model structural error.Citation: Thyer, M., B. Renard, D. Kavetski, G. Kuczera, S. W. Franks, and S. Srikanthan (2009), Critical evaluation of parameter consistency and predictive uncertainty in hydrological modeling: A case study using Bayesian total error analysis, Water Resour. Res.,

Calibration of hydrological models using flow-duration curves

“…A perfect match of 100 % can then not be achieved if the simulated uncertainty is overestimated. More complex measures, such as a PQQ-plot (Thyer et al, 2009) or a rank histogram, analyse the quantiles of the observed value in the simulated distribution. The generalised rank histogram (McMillan et al, 2010) is an extension of the rank histogram that compares two uncertain distributions so that uncertainty in the observed data can be accounted for.…”

“…Uncertainty in discharge data, which has been shown to be sometimes substantial (Di Baldassarre and Montanari, 2009;Pelletier, 1988;Krueger et al, 2010;PetersenOverleir et al, 2009) and influence the calibration of hydrological models (McMillan et al, 2010;Aronica et al, 2006), is usually not accounted for in model evaluation with traditional performance measures. Novel approaches in environmental modelling that include evaluation-data uncertainty in model calibration include Bayesian calibration to an estimated probability-density function of discharge (McMillan et al, 2010), Bayesian calibration with a simplified error model (Huard and Mailhot, 2008;Thyer et al, 2009), fuzzy rule based performance measures (Freer et al, 2004) and limits-of-acceptability calibration in GLUE for rainfallrunoff modelling (Liu et al, 2009), flood mapping (Pappenberger et al, 2007), environmental tracer modelling (Page et al, 2007) and flood-frequency estimation (Blazkova and Beven, 2009). Here we explore the limits-of-acceptability GLUE approach applied to flow-duration curves, which could be a way of dealing with some of the effects of nonstationary epistemic errors on the identification of feasible model parameters in real applications (Beven, 2006(Beven, , 2010Beven and Westerberg, 2011;Beven et al, 2008).…”

Quantile hydrologic model selection and model structure deficiency assessment: 1. Theory

“…The underlying motivation is to compare two model structures in terms of its deficiencies in representing the underlying processes (''truth''). In contrast to Bayesian approaches to model selection [such as Kavetski et al, 2006;Thyer et al, 2009;Schoups and Vrugt, 2010] where various sources of errors can be explicitly modeled, no assumption on the cumulative distribution of the residuals is made, where a distribution of residuals is due to unknown measurement errors and model structure deficiency.…”

“…These methods are powerful and yield useful insights for improving model structures. A validation of assumptions is generally made by Q-Q plots by mapping observed quantiles to prediction quantiles for a variable of interest [Thyer et al, 2009;Schoups and Vrugt, 2010]. A Q-Q plot verifies whether the prediction quantiles follow the observed quantiles, thereby assessing the applicability of the model assumptions.…”

“…[2] Current practice of uncertainty assessment of hydrologic models hypothesizes potential sources of errors, assumes that it obey certain distribution types and nests these distributions within a Bayesian inference framework [Kavetski et al, 2006;Thyer et al, 2009;Schoups and Vrugt, 2010;Smith et al, 2010]. Bayesian inference therefore allows simultaneous modeling of uncertainties due to model and measurement errors.…”