Uncertainty Quantification in Machine Learning Modeling for Multi-Step Time Series Forecasting: Example of Recurrent Neural Networks in Discharge Simulations

Abstract

As a revolutionary tool leading to substantial changes across many areas, Machine Learning (ML) techniques have obtained growing attention in the field of hydrology due to their potentials to forecast time series. Moreover, a subfield of ML, Deep Learning (DL) is more concerned with datasets, algorithms and layered structures. Despite numerous applications of novel ML/DL techniques in discharge simulation, the uncertainty involved in ML/DL modeling has not drawn much attention, although it is an important issue. In this study, a framework is proposed to quantify uncertainty contributions of the sample set, ML approach, ML architecture and their interactions to multi-step time-series forecasting based on the analysis of variance (ANOVA) theory. Then a discharge simulation, using Recurrent Neural Networks (RNNs), is taken as an example. Long Short-Term Memory (LSTM) network, a state-of-the-art DL approach, was selected due to its outstanding performance in time-series forecasting, and compared with simple RNN. Besides, novel discharge forecasting architecture is designed by combining the expertise of hydrology and stacked DL structure, and compared with conventional design. Taking hourly discharge simulations of Anhe (China) catchment as a case study, we constructed five sample sets, chose two RNN approaches and designed two ML architectures. The results indicate that none of the investigated uncertainty sources are negligible and the influence of uncertainty sources varies with lead-times and discharges. LSTM demonstrates its superiority in discharge simulations, and the ML architecture is as important as the ML approach. In addition, some of the uncertainty is attributable to interactions rather than individual modeling components. The proposed framework can both reveal uncertainty quantification in ML/DL modeling and provide references for ML approach evaluation and architecture design in discharge simulations. It indicates uncertainty quantification is an indispensable task for a successful application of ML/DL.

Assessment of Impacts of Climate Change on Indian Riverine Thermal Regimes Using Hybrid Deep Learning Methods

“…To address such model uncertainties originating from various DL, the ensemble of DL models, stacking algorithms, etc. (Piotrowski et al., 2021; Song et al., 2020) can be adopted which can combine RWT predictions from various DL models. Furthermore, such ensemble/stacking algorithms allow the decision makers to choose the best possible prediction within a range of predictions (Rehana & Mujumdar, 2014).…”

Impact of climate change on river water temperature and dissolved oxygen: Indian riverine thermal regimes

“…To address such model uncertainties, the ensemble of DL models, etc. 79 , 80 can be adopted which can combine RWT predictions from various DL models, allowing the decision-makers to choose the best possible prediction within a range of predictions 81 . Despite the effectiveness of the modeling frameworks, as demonstrated in the present work, it has some limitations.…”

A Forecasting Model in Managing Future Scenarios to Achieve the Sustainable Development Goals of Thailand’s Environmental Law: Enriching the Path Analysis-Varima-Ovi Model

“…Geysen et al (2018) validated operational thermal load forecasting in district heating networks with the use of machine learning and expert system; linear regression, extremely randomized trees regression, feed-forward neural network and support vector machine. Lastly, Song et al (2020) proposed a framework to quantify uncertainty in machine learning (ML) modelling in order to forecast multistep time-series using the analysis of variance (ANOVA) theory.…”