Appraisal of data-driven and mechanistic emulators of nonlinear simulators: The case of hydrodynamic urban drainage models

Abstract

Many model based scientific and engineering methodologies, such as system identification, sensitivity analysis, optimization and control, require a large number of model evaluations. In particular, model based real-time control of urban water infrastructures and online flood alarm systems require fast prediction of the network response at different actuation and/or parameter values. General purpose urban drainage simulators are too slow for this application. Fast surrogate models, so-called emulators, provide a solution to this efficiency demand. Emulators are attractive, because they sacrifice unneeded accuracy in favor of speed. However, they have to be fine-tuned to predict the system behavior satisfactorily. Also, some emulators fail to extrapolate the system behavior beyond the training set. Although, there are many strategies for developing emulators, up until now the selection of the emulation strategy remains subjective. In this paper, we therefore compare the performance of two families of emulators for open channel flows in the context of urban drainage simulators. We compare emulators that explicitly use knowledge of the simulator's equations, i.e. mechanistic emulators based on Gaussian Processes, with purely data-driven emulators using matrix factorization. Our results suggest that in many urban applications, naive data-driven emulation outperforms mechanistic emulation. Nevertheless, we discuss scenarios in which we think that mechanistic emulation might be favorable for i) extrapolation in time and ii) dealing with sparse and unevenly sampled data. We also provide many references to advances in the field of Machine Learning that have not yet permeated into the Bayesian environmental science community. arXiv:1609.08395v2 [stat.ME]

Accelerating Bayesian inference in hydrological modeling with a mechanistic emulator

“…Depending on the simulation model, the mechanistic emulator could be replaced by a purely data-driven surrogate with similar or even better performance (Carbajal et al, 2017) surrogates. In this paper, we restrict ourselves to mechanistic emulators.…”

“…But it is advantageous to employ some knowledge about parameter dependence with our heuristic release rate, which doesnt express much more than an increase of the release rate if, on average, slopes are steeper, overland flow paths are wider, etc. Data-driven methods of establishing optimal mappings between simulator and emulator parameters have been explored in Carbajal et al (2017).…”

A Data-Driven Surrogate Modelling Approach for Acceleration of Short-Term Simulations of a Dynamic Urban Drainage Simulator

“…Over recent years, a considerable part of research in the field of surrogate modelling for urban water simulators has emphasized the use of data-driven approaches; such as Artificial Neural Networks [16], Neuro-fuzzy Systems [17], Deep Learning [18], Radial Basis Functions [19], Kriging [4], Polynomials [20], and Gaussian Processes Emulators (GPEs) [21]. The main reason for popularity of these methods is their generic nature, in which there is no, or little, need to deal with the mathematics behind the simulators.…”

“…An earlier version of that in reference [3] with application to shallow water equations can be seen in reference [25]. A key study exists comparing the mechanism-based GPE with a purely data-driven GPE [21], in which, it was asserted that data-driven GPE outperforms the mechanistic one in many applications. Appraisal of data-driven GPE methods, from the prediction accuracy point of view, over other emulation techniques such as linear model (LM), generalized additive models (GAMs) and random forests (RFs), is highlighted in reference [26] as well.…”

Quantifying Roughness Coefficient Uncertainty in Urban Flooding Simulations through a Simplified Methodology

“…In those cases, it is possible to propose a functional representation of the model, which approximates its behaviour with respect to the parameter values. This is often referred to as surrogate or meta-modelling [33][34][35]. A polynomial orthogonal expansion is a method to create surrogate structure for space-time-dependent models [16].…”