Treatment of uncertainty using ensemble methods: Comparison of sequential data assimilation and Bayesian model averaging

Abstract

[1] Predictive uncertainty analysis in hydrologic modeling has become an active area of research, the goal being to generate meaningful error bounds on model predictions. State-space filtering methods, such as the ensemble Kalman filter (EnKF), have shown the most flexibility to integrate all sources of uncertainty. However, predictive uncertainty analyses are typically carried out using a single conceptual mathematical model of the hydrologic system, rejecting a priori valid alternative plausible models and possibly underestimating uncertainty in the model itself. Methods based on Bayesian model averaging (BMA) have also been proposed in the statistical and meteorological literature as a means to account explicitly for conceptual model uncertainty. The present study compares the performance and applicability of the EnKF and BMA for probabilistic ensemble streamflow forecasting, an application for which a robust comparison of the predictive skills of these approaches can be conducted. The results suggest that for the watershed under consideration, BMA cannot achieve a performance matching that of the EnKF method.

Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models

“…At first glance this suggests there is actually limited potential to use outputs from our multimodel configuration as an estimate of model uncertainty. Ideally, simulations of streamflow from different model structures will bracket the observed streamflow [e.g., Georgakakos et al, 2004;Vrugt and Robinson, 2007]; when this does not occur (as in this study) it can be viewed as being indicative of a lack of independent information in different models. However, the consistency in model errors may also arise because errors in model inputs affect different models in similar ways.…”

Quantitative comparison of canopy conductance models using a Bayesian approach

“…Consequently, different models provide different estimates of system response, which may lead to different predictions and inferences regarding system functions [e.g., Pan et al, 1998;Cramer et al, 1999;Luckai and Larocque, 2002;Adams et al, 2004]. The inability to identify a unique model structure (i.e., structural uncertainty) out of the various possibilities is often taken into account in model prediction and the estimate of prediction uncertainty by using multiple models in an ensemble in methodologies such as Bayesian Model Averaging (BMA) [e.g., Neuman, 2003;Ye et al, 2004;Raftery et al, 2005;Ajami et al, 2007;Vrugt and Robinson, 2007]. However, an important objective of using semiempirical models in the analysis of complex Earth systems is to be able to make inferences about system processes.…”

Increasing parameter certainty and data utility through multi-objective calibration of a spatially distributed temperature and solute model

“…This type of flexibility is not available when optimization algorithms rely solely on the options encoded to solve the problem, which is the case for most single objective algorithms (e.g., nonlinear gradient-based search algorithms such as the Levenberg-Marquardt algorithm (Marquardt, 1963, used by Hil, 1998Doherty, 2005;Poeter et al, 2005), evolutionary algorithms (Duan et al, 1992;Deb, 2001) or Bayesian approaches (Metropolis et al, 1953;Hastings, 1970;Doherty, 2003). Although multi-objective algorithms (e.g., Gupta et al, 1998;Boyle et al, 2000;Madsen, 2000Madsen, , 2003Madsen et al, 2002;Deb et al, 2002;Vrugt et al, 2003a,b) and multi algorithm genetically adaptive search methods (AMALGAM, Vrugt and Robinson, 2007) incorporate multiple datasets into optimization, the number of datasets considered have generally been limited to two or three time series and there is limited flexibility in the objectives considered due to limitations of the algorithm design requirements (e.g., soil hydraulic models calibrated to multiple soil depths, but only at one location; Wöhliing et al, 2008).…”

“…Additionally, many hydrological modeling procedures do not make the best use of available information (Wagener et al, 2001). Current research on the calibration problem primarily focuses on uncertainty analysis and consideration of multiple objectives (Fu and Gomez-Hernandez, 2009;Blasone et al, 2008;Ajami et al, 2007;Duan et al, 2007;Vrugt and Robinson, 2007). Rather than selecting a single preferred parameter set, equifinality of models recognizes that there may be no single, correct set of parameter values for a given model and that different parameter sets may give acceptable model performance (Beven, 2001).…”