Benchmarking Data-Driven Rainfall-Runoff Models in Great Britain: A comparison of LSTM-based models with four lumped conceptual models

Abstract

Abstract. Long short-term memory models (LSTMs) are recurrent neural networks from the emerging field of Deep Learning (DL), which have shown recent promise when predicting time-series especially when data are abundant. Rainfall-runoff modelling presents a challenge, yet accurate hydrological models are vital for flood forecasting, hazard impact assessment, and to assess the potential effects of climate change on floods and water resources. In this study, we compare the performance of two DL-based models, a LSTM and an Entity Aware LSTM (EA LSTM). The DL models were trained using a newly published data set, CAMELS-GB, for a sample of 518 catchments across Great Britain. To identify spatial and seasonal patterns in model performance, we compare the DL models against benchmark outputs from four lumped conceptual models recently configured for rainfall-runoff modelling in Great Britain. Our findings show that the LSTM models simulate discharge with consistently high model performance scores, including in catchments typically considered difficult to model. The LSTM achieves a mean catchment NSE of 0.88 (0.86 for the EALSTM), which represents a performance improvement of 10 %–16 % compared with the benchmark conceptual models. Seasonal and spatial patterns indicate that the largest performance improvement relative to the benchmark is in the drier summer months and in drier catchments in the South East of England. By comparing LSTMs with conceptual models, we diagnose possible reasons for their different performance. We suggest that LSTMs offer useful predictive capability for rainfall-runoff modelling in Great Britain and elsewhere and note their value to support process understanding in locations where processes are less well understood.

How can regime characteristics of catchments help in training of local and regional LSTM-based runoff models?

“…catchment attributes) features outperformed the regional LSTM model that did not take into account any static features as well as all tested local benchmark models. In the climatic context of Great Britain and on a sample of 518 catchments, Lees et al (2021) observed also an outperformance of regional LSTM models over four conceptual benchmark models, namely SACRAMENTO, ARNOVIC, TOPMODEL and PRMS. These studies (Kratzert et al, 2018(Kratzert et al, , 2019aLees et al, 2021) show that regional training of the LSTM brings performance improvement when compared with local training since regional models learn better.…”

“…In the climatic context of Great Britain and on a sample of 518 catchments, Lees et al (2021) observed also an outperformance of regional LSTM models over four conceptual benchmark models, namely SACRAMENTO, ARNOVIC, TOPMODEL and PRMS. These studies (Kratzert et al, 2018(Kratzert et al, , 2019aLees et al, 2021) show that regional training of the LSTM brings performance improvement when compared with local training since regional models learn better. It is therefore tried to improve learning by increasing the number of train catchments.…”

“…In the previous studies by Kratzert et al (2018) and Lees et al (2021), the length of the input sequence, i.e. the LSTM's window size for looking to the past and hereafter called lookback, was set to 365 [day] so that the dynamics of a full annual cycle could be captured.…”

Comment on hess-2021-127

“…The Nash-Sutcliffe Efficiency (NSE) (Nash and Sutcliffe, 1970), Equation 13is perhaps the most widely used performance measure in hydrology (Ewen, 2011). It has been used for many years and there is extensive literature discussing its strengths and weaknesses (Gupta et al, 2009). Owing to the squared term in the definition of NSE, it is more heavily influenced by high flows.…”

“…and a variability (SDError Equation 16) term (Gupta et al, 2009). The bias term measures the error in predicting the mean flow.…”