A methodological framework for the evaluation of short-range flash-flood hydrometeorological forecasts at the event scale

Abstract

Abstract. This paper presents a methodological framework designed for the event-based evaluation of short-range hydro-meteorological ensemble forecasts, in the specific context of an intense flash-flood event characterized by high spatio-temporal variability. The proposed evaluation adopts the point of view of end-users in charge of the organization of evacuations and rescue operations at a regional scale. Therefore, the local exceedance of discharge thresholds should be anticipated in time and accurately localized. A step-by-step approach is proposed, including first an evaluation of the rainfall forecasts. This first step helps to define appropriate spatial and temporal scales for the evaluation. The anticipation of hydrological responses (discharge thresholds) is then analyzed, with a focus on the flood rising limb period, and using simulated flows and zero-future rainfall forecasts as references. Based on this second step, several gauged sub-catchments are selected, at which a detailed evaluation of the forecast hydrographs is finally achieved. This methodology is tested and illustrated on the October 2018 flash-flood, which affected part of the Aude River basin (south-eastern France). Three ensemble rainfall now casting research products recently proposed by Météo-France are evaluated and compared. The results confirm the added value of these ensembles to improve discharge threshold detection and anticipation, but at the price of a significant increase of false alarms. Finally, even if the evaluation of ensemble hydro-meteorological forecasts based on a low number of documented flood events remains challenging due to the limited statistical representation of the available data, the evaluation framework proposed herein should contribute to draw robust conclusions about the usefulness of newly developed rainfall ensemble forecast approaches, and about their limits and possible improvements.

Assessing the ability of a new seamless short-range ensemble rainfall product to anticipate flash floods in the French Mediterranean area

“…Contingency tables are usually filled by combining a continuous temporal sequence of forecasts, at one single site and for one unique lead time. However, we followed here the principle proposed by Charpentier-Noyer et al (2022) to build the contingency tables by aggregating the forecasts issued during the most critical phase of the event (i.e. the forecasts issued during the flood rising limb, just before the threshold exceedance by Q sim , independently of the lead time).…”

“…However, the POFD score is sensitive to the extent of the verification area, which directly determines the number of correct rejections (see figure 6). The choice of the verification area was already identified as an important issue by Charpentier-Noyer et al (2022) who suggested to pay a particular attention to the choice of the "HFA" (Hydrological Focus Area). For that reason, we chose to summary the contingency tables based on the Critical Success Index (CSI = a a+b+c ) instead of POD/POFDs.…”

“…Maximizing the anticipation times of good forecasts is another desirable property in an operational forecasting context, but anticipation has not been evaluated in our work. A estimation of the anticipation times involved in the contingency tables values was attempted by Charpentier-Noyer et al (2022). Unfortunately, our present use of a forecast-based stratification problem following Bellier et al (2017), would make it more difficult to investigate the notion of anticipation.…”

“…This is achieved by comparing the hydrological forecasts obtained using QPFs with simulated discharges (i.e. based on QPEs), at each pixel of the hydrological model grid (1-km resolution), and following a methodology adapted from Charpentier-Noyer et al (2022).…”

Assessing the ability of a seamless short-range ensemble rainfall product to detect flash floods on the French Mediterranean area

“…La méthode d'évaluation appliquée porte sur la capacité de la chaine de prévision à anticiper les dépassements de seuils de débits de périodes de retour 2, 5 et 10 ans, pour huit événements de crues soudaines récents (entre 2019 et 2021) qui ont été rejoués. L'évaluation se base sur l'utilisation de tableaux de contingence et leur représentation cartographique, suivant la méthode proposée par M. Charpentier-Noyer et al [5] qui a été légèrement adaptée ici, ainsi que sur le calcul de scores synthétiques (Critical Success Index). Il est important de noter que cette évaluation, bien que basée sur des scores statistiques classiques, ne permet pas de caractériser la performance des prévisions sur de longues chroniques temporelles, mais simplement de regarder leur capacité à prédire correctement des événements spécifiques de forte intensité.…”