Toward diagnostic model calibration and evaluation: Approximate Bayesian computation

Abstract

[1] The ever increasing pace of computational power, along with continued advances in measurement technologies and improvements in process understanding has stimulated the development of increasingly complex hydrologic models that simulate soil moisture flow, groundwater recharge, surface runoff, root water uptake, and river discharge at different spatial and temporal scales. Reconciling these high-order system models with perpetually larger volumes of field data is becoming more and more difficult, particularly because classical likelihood-based fitting methods lack the power to detect and pinpoint deficiencies in the model structure. Gupta et al. (2008) has recently proposed steps (amongst others) toward the development of a more robust and powerful method of model evaluation. Their diagnostic approach uses signature behaviors and patterns observed in the input-output data to illuminate to what degree a representation of the real world has been adequately achieved and how the model should be improved for the purpose of learning and scientific discovery. In this paper, we introduce approximate Bayesian computation (ABC) as a vehicle for diagnostic model evaluation. This statistical methodology relaxes the need for an explicit likelihood function in favor of one or multiple different summary statistics rooted in hydrologic theory that together have a clearer and more compelling diagnostic power than some average measure of the size of the error residuals. Two illustrative case studies are used to demonstrate that ABC is relatively easy to implement, and readily employs signature based indices to analyze and pinpoint which part of the model is malfunctioning and in need of further improvement.

Markov chain Monte Carlo simulation using the DREAM software package: Theory, concepts, and MATLAB implementation

“…Thus, the four summary metrics used herein contain sufficient information to provide a reasonably adequate calibration. The interested reader is referred to Vrugt and Sadegh (2013) and Vrugt (submitted for publication) for a much more detailed ABC analysis with particular focus on diagnosis and detection of epistemic errors.…”

“…A plea for this approach has been made by Gupta et al (2008) and Vrugt and Sadegh (2013) have provided the mathematical foundation for diagnostic model evaluation using ABC. Subsequent work by Sadegh et al (2015b) has shown the merits of this methodology by addressing the stationarity paradigm.…”

“…Note that model structural error (label 4) (also called epistemic error) has received relatively little attention, but is key to learning and scientific discovery (Vrugt et al, 2005;Vrugt and Sadegh, 2013).…”

Parameter-state ensemble data assimilation using Approximate Bayesian Computing for short-term hydrological prediction

“…As described by Vrugt and Sadegh (2013), for scenarios focused on parameter uncertainty (as is the case in this paper) the posterior parameter distribution p(θ|y) given the streamflow y is estimated using Bayes theorem:…”

Quantifying Precipitation Uncertainty for Land Data Assimilation Applications

“…In particular, a Bayesian approach allows us to update prior ensembles using measurements and to investigate the associated uncertainties (Bocquet et al 2010). In the recent decade, Bayesian-based approaches have been widely used to estimate hydrologic variables, especially for probabilistic hydrologic forecasting (Kavetski et al 2006;Bulygina and Gupta 2010;Renard et al 2011;Moradkhani et al 2012;DeChant and Moradkhani 2012;Najafi et al 2012;Vrugt and Sadegh 2013;DeChant and Moradkhani 2014). All of these studies have been focused on quantifying scalar variables or low-dimensional parameter spaces.…”