Benchmarking of a Physically Based Hydrologic Model

Abstract

The concepts of model benchmarking, model agility, and large-sample hydrology are becoming more prevalent in hydrologic and land surface modeling. As modeling systems become more sophisticated, these concepts have the ability to help improve modeling capabilities and understanding. In this paper, their utility is demonstrated with an application of the physically based Variable Infiltration Capacity model (VIC). The authors implement VIC for a sample of 531 basins across the contiguous United States, incrementally increase model agility, and perform comparisons to a benchmark. The use of a large-sample set allows for statistically robust comparisons and subcategorization across hydroclimate conditions. Our benchmark is a calibrated, time-stepping, conceptual hydrologic model. This model is constrained by physical relationships such as the water balance, and it complements purely statistical benchmarks due to the increased physical realism and permits physically motivated benchmarking using metrics that relate one variable to another (e.g., runoff ratio). The authors find that increasing model agility along the parameter dimension, as measured by the number of model parameters available for calibration, does increase model performance for calibration and validation periods relative to less agile implementations. However, as agility increases, transferability decreases, even for a complex model such as VIC. The benchmark outperforms VIC in even the most agile case when evaluated across the entire basin set. However, VIC meets or exceeds benchmark performance in basins with high runoff ratios (greater than ~0.8), highlighting the ability of large-sample comparative hydrology to identify hydroclimatic performance variations.

A Ranking of Hydrological Signatures Based on Their Predictability in Space

“…We ask whether explicitly accounting for hydrological processes (instead of adopting a purely statistical, data‐driven approach) improves the signature predictions. There is a growing recognition of the utility of benchmarks in hydrology (Best et al, ; Newman et al, ; Seibert, ; Seibert et al, ), we use these hydrological simulations to evaluate the predictions of our random forests, keeping in mind that random forests are significantly quicker to setup and run than calibrated hydrological models. Why are some signatures better regionalized than others? It is known that the accuracy of the regionalization varies from signature to signature, but it is unclear why this is the case.…”

On hypothesis testing in hydrology: Why falsification of models is still a really good idea

“…Thus, any process model that results in an entropy greater than the data‐based model could be rejected (though it is important to compare like with like: in an early application of this approach Gong, Gupta, Yang, Sricharan, and Hero () tested a hydrological simulation model against a one‐step ahead data‐based forecasting model; unsurprisingly, the simulation model did not perform as well!). Some interesting recent studies have concerned “model benchmarking” in evaluating land surface parameterizations in climate models (Best et al, ; Haughton et al, ; Nearing, Mocko, Peters‐Lidard, Kumar, & Xia, ) and in a multisite application of the VIC rainfall‐runoff model (Newman et al ()).…”

On the choice of calibration metrics for high flow estimation using hydrologic models

“…The VIC model, which includes explicit soil-vegetation-snow processes, has been used for a wide range of hydrologic applications, and has recently been evaluated in large-sample predictability benchmark studies (Newman et al, 2017). The mHM model has been shown to provide robust hydrologic simulations over both Europe and the US (Kumar et al, 2013a;Rakovec et al, 2016b) and is currently being used in application studies (e.g., Thober et al, 2018;Samaniego et al, 2018).…”

“…The model parameters calibrated for each model are the same as previously discussed: VIC (Newman et al, 2017; and mHM (Rakovec et al, 2016a, b). Although alternative calibration parameter sets have also been used by others, particularly for VIC (Newman et al, 2017), the purpose of this study is purely to examine the effect of performance metrics used for calibration, not to obtain "optimal" parameter sets.…”

“…Although alternative calibration parameter sets have also been used by others, particularly for VIC (Newman et al, 2017), the purpose of this study is purely to examine the effect of performance metrics used for calibration, not to obtain "optimal" parameter sets. Each model was identically configured for each of the 492 basins, and both models used the same set of underlying physiographical and meteorological datasets, so that performance differences can be attributed mainly to the strategy used to obtain the parameter estimates.…”