Use of paired simple and complex models to reduce predictive bias and quantify uncertainty

Abstract

[1] Modern environmental management and decision-making is based on the use of increasingly complex numerical models. Such models have the advantage of allowing representation of complex processes and heterogeneous system property distributions inasmuch as these are understood at any particular study site. The latter are often represented stochastically, this reflecting knowledge of the character of system heterogeneity at the same time as it reflects a lack of knowledge of its spatial details. Unfortunately, however, complex models are often difficult to calibrate because of their long run times and sometimes questionable numerical stability. Analysis of predictive uncertainty is also a difficult undertaking when using models such as these. Such analysis must reflect a lack of knowledge of spatial hydraulic property details. At the same time, it must be subject to constraints on the spatial variability of these details born of the necessity for model outputs to replicate observations of historical system behavior. In contrast, the rapid run times and general numerical reliability of simple models often promulgates good calibration and ready implementation of sophisticated methods of calibration-constrained uncertainty analysis. Unfortunately, however, many system and process details on which uncertainty may depend are, by design, omitted from simple models. This can lead to underestimation of the uncertainty associated with many predictions of management interest. The present paper proposes a methodology that attempts to overcome the problems associated with complex models on the one hand and simple models on the other hand, while allowing access to the benefits each of them offers. It provides a theoretical analysis of the simplification process from a subspace point of view, this yielding insights into the costs of model simplification, and into how some of these costs may be reduced. It then describes a methodology for paired model usage through which predictive bias of a simplified model can be detected and corrected, and postcalibration predictive uncertainty can be quantified. The methodology is demonstrated using a synthetic example based on groundwater modeling environments commonly encountered in northern Europe and North America.Citation: Doherty, J., and S. Christensen (2011), Use of paired simple and complex models to reduce predictive bias and quantify uncertainty, Water Resour. Res., 47, W12534,

Testing alternative uses of electromagnetic data to reduce the prediction error of groundwater models

“…As discussed by Doherty and Christensen (2011), if the plotted data do not scatter around the identity line, it indicates bias in the model prediction. If the intercept of a regression line through the scatter of points deviates from zero, it indicates consistent bias in the prediction due to consistent errors in null space parameter components omitted from the parameterized groundwater model; if the slope of the regression line deviates from unity, it indicates parameter surrogacy incurred through model calibration (see Doherty and Christensen (2011) for further explanation).…”

“…The synthetic demonstration model used here is, to a large degree, inspired by the model of Doherty and Christensen (2011). The hydrogeological setting of the model domain is typical for large areas of northern Europe and North America: a glacially formed landscape with a buried tunnel valley eroded into impermeable bedrock (fat clay) with very low electrical resistivity (Wright, 1973;Piotrowski, 1994;Clayton et al, 1999;Jørgensen and Sandersen, 2006).…”

“…The deposits above the bedrock are glacial of different types. For the sake of clarity, the synthetic model will be described in the section below, and the exceptions and changes from the setup of Doherty and Christensen (2011) will be highlighted. Each HYTEB step will be presented in order following Fig.…”

“…A groundwater model will therefore always contain small-scale structural errors, which may not cause bias in predictions but may still cause large prediction uncertainty (Cooley, 2004;Cooley and Christensen, 2006;Refsgaard et al, 2012). (iii) A model is also prone to possessing largescale structural errors that can cause significant bias and uncertainty of estimated parameters and simulated predictions (Doherty and Welter, 2010;Doherty and Christensen, 2011;Refsgaard et al, 2012). This bias and uncertainty can be reduced by collecting data that resolve the large-scale structures of the studied hydrogeological system, which can then be accurately represented in the model.…”

Novel Two-Stage Method for Low-Order Polynomial Model

“…Another increasingly popular usage for metamodels is to deal with missing data and gain insight into the contributions of each input variable to associated output variables [4]. Besides, metamodels can also be used to reduce numerical instability [5]. Moreover, metamodels can be utilized as calibration methods for low-fidelity simulations of limited accuracy [6].…”