On the difficulty to optimally implement the Ensemble Kalman filter: An experiment based on many hydrological models and catchments

Abstract

Forecast reliability and accuracy is a prerequisite for successful hydrological applications. This aim may be attained by using data assimilation techniques such as the popular Ensemble Kalman filter (EnKF). Despite its recognized capacity to enhance forecasting by creating a new set of initial conditions, implementation tests have been mostly carried out with a single model and few catchments leading to case specific conclusions. This paper performs an extensive testing to assess ensemble bias and reliability on 20 conceptual lumped models and 38 catchments in the Province of Québec with perfect meteorological forecast forcing. The study confirms that EnKF is a powerful tool for short range forecasting but also that it requires a more subtle setting than it is frequently recommended. The success of the updating procedure depends to a great extent on the specification of the hyper-parameters. In the implementation of the EnKF, the identification of the hyper-parameters is very unintuitive if the model error is not explicitly accounted for and best estimates of forcing and observation error lead to overconfident forecasts. It is shown that performance are also related to the choice of updated state variables and that all states variables should not systematically be updated. Additionally, the improvement over the open loop scheme depends on the watershed and hydrological model structure, as some models exhibit a poor compatibility with EnKF updating. Thus, it is not possible to conclude in detail on a single ideal manner to identify an optimal implementation; conclusions drawn from a unique event, catchment, or model are likely to be misleading since transferring hyper-parameters from a case to another may be hazardous. Finally, achieving reliability and bias jointly is a daunting challenge as the optimization of one score is done at the cost of the other.

On the assimilation set-up of ASCAT soil moisture data for improving streamflow catchment simulation

“…In the EnKF, the background state vector becomes an n (state variables) x N r (ensemble members) background matrix (X b ) needed to derive the model error matrix (Thiboult and Anctil, 2015), where:…”

“…6), at each time step k, represents the relative importance of the observation error with respect to the prior estimate (i.e., model simulation) and acts as a weighting M A N U S C R I P T coefficient. Z k denotes the covariance of the observational noise (Thiboult and Anctil, 2015) at time step k, P k accounts for model uncertainty, and H k is the observation operator (SWI vector in this study). G k is thus calculated as:…”

Moving beyond the cost–loss ratio: economic assessment of streamflow forecasts for a risk-averse decision maker

“…In this study, data assimilation is a necessity rather than a choice and is not at all the primary objective. For this reason, the limits of the above-mentioned distributions were not optimised as in Thiboult and Anctil (2015). Those limits were fixed according to the guidelines in (Mamono, 2010) and (Abaza et al, 2015) and the experience gained during manual data assimilation.…”

The HOOPLA toolbox: a HydrOlOgical Prediction LAboratory to explore ensemble rainfall-runoff modeling

“…This suggests that an adequate data assimilation schemes may help compensate for a calibration that sought minimizing the error over an entire chronicle. This gain varies according to catchments, but also to models to a lesser extent (Thiboult and Anctil, 2015). https://doi.org/10.5194/hess-2020-6 Preprint.…”