Testing hydrological models under changing conditions is essential to evaluate their ability to cope with changing catchments and their suitability for impact studies. With this perspective in mind, a workshop dedicated to this issue was held at the 2013 General Assembly of the International Association of Hydrological Sciences (IAHS) in Göteborg, Sweden, in July 2013, during which the results of a common testing experiment were presented. Prior to the workshop, the participants had been invited to test their own models on a common set of basins showing varying conditions specifically set up for the workshop. All these basins experienced changes, either in physical characteristics (e.g. changes in land cover) or climate conditions (e.g. gradual temperature increase). This article presents the motivations and organization of this experiment-that is-the testing (calibration and evaluation) protocol and the common framework of statistical procedures and graphical tools used to assess the model performances. The basins datasets are also briefly introduced (a detailed description is provided in the associated Supplementary material).

Abstract. As all hydrological models are intrinsically limited hypotheses on the behaviour of catchments, models -which attempt to represent real-world behaviour -will always remain imperfect. To make progress on the long road towards improved models, we need demanding tests, i.e. true crash tests. Efficient testing requires large and varied data sets to develop and assess hydrological models, to ensure their generality, to diagnose their failures, and ultimately, help improving them.

All that glitters is not gold is one of those universal truths that also applies to hydrology, and particularly to the issue of model calibration, where a glittering mathematical optimum is too often mistaken for a hydrological optimum. This commentary aims at underlining the fact that calibration difficulties have not disappeared with the advent of the latest search algorithms. While it is true that progress on the numerical front has allowed us to quasi-eradicate miscalibration issues, we still too often underestimate the remaining hydrological task: screening mathematical optima in order to identify those parameter sets which will also work sufficiently outside the calibration period.

Abstract. This paper compares event-based and continuous hydrological modelling approaches for real-time forecasting of river flows. Both approaches are compared using a lumped hydrologic model (whose structure includes a soil moisture accounting (SMA) store and a routing store) on a data set of 178 French catchments. The main focus of this study was to investigate the actual impact of soil moisture initial conditions on the performance of flood forecasting models and the possible compensations with updating techniques. The rainfall-runoff model assimilation technique we used does not impact the SMA component of the model but only its routing part. Tests were made by running the SMA store continuously or on event basis, everything else being equal. The results show that the continuous approach remains the reference to ensure good forecasting performances. We show, however, that the possibility to assimilate the last observed flow considerably reduces the differences in performance. Last, we present a robust alternative to initialize the SMA store where continuous approaches are impossible because of data availability problems.

Abstract. This paper compares event-based and continuous hydrological modelling approaches for real-time forecasting of river flows. Both approaches are compared using a lumped hydrologic model (whose structure includes a soil moisture accounting (SMA) store and a routing store) on a data set of 178 French catchments. The main focus of this study was to investigate the actual impact of soil moisture initial conditions on the performance of flood forecasting models and the possible compensations with updating techniques. The rainfall runoff model assimilation technique we used does not impact the SMA component of the model but only its routing part. Tests were made by running the SMA store continuously or on event basis, everything else being equal. The results show that the continuous approach remains the reference to ensure good forecasting performances. We show, however, that the possibility to assimilate the last observed flow considerably reduces the differences in performance. Last, we present a robust alternative to initialize the SMA store where continuous approaches are impossible because of data availability problems.

As all hydrological models are intrinsically limited hypotheses on the behaviour of catchments, models-which attempt to represent real-world behaviour-will always remain imperfect. To make progress on the long road towards improved models, we need demanding tests, i.e. true crash tests. Efficient testing requires large and varied data sets to develop and assess hydrological models, to ensure their generality, to diagnose their failures, and ultimately, help improving them.

Abstract. Flood forecasting uncertainty is crucial information for decision makers. However, deterministic only forecasts have been communicated in France until now, like in many other countries. The French Flood Forecast Centres (FFCs) recently set up a new service which aims at publishing quantitative forecasts along with their associated uncertainty. Two surveys (one of the greater audience and identified end-users, another of FFCs worldwide) were conducted to design it. The forecasters' toolbox was then supplemented with two new tools. The first one provides automatic forecasting uncertainty estimations calibrated on past forecasting error series. The second one allows the forecasters to incorporate their own expertise to adjust the automatically calculated uncertainty estimation. The evaluation of the forecast uncertainty estimations issued in real time in 2014 suggests that even if these assessments are perfectible, they are already informative and useful for end-users. The first feedbacks from forecasters and crisis managers also show that if the assessment of probabilistic forecasts remains a technical challenge, their use is foremost a human challenge. This move is a paradigm change for both forecasters and decision makers. Therefore, they have to be accompanied in order to achieve this deep shift in their professional practice.

Quadratic criteria (i.e. based on squared residuals) are widely used to assess the performance of hydrological models. However, the largest errors have a relatively strong influence on the final criterion values, which may be considered a drawback for a complete assessment. This paper studies the case of updated models used for real-time forecasting. It is shown that the fraction of the data series actually impacting the final criterion value is small on many catchments and corresponds to the time steps characterised by the greatest runoff variations. In fact, model updating makes the error distribution more peak-shaped, giving even more relative importance to the time steps with the largest errors. Therefore, assessing the performance of an updated model with a quadratic criterion emphasises that these criteria focus more on the most difficult time steps to model (and the most interesting ones in the case of short-term flood forecasting).Key words model assessment; skill scores; quadratic criteria; flood forecasting; persistence index; heteroscedasticity Quelle signification accorder aux critères quadratiques? Partie 2. De la contribution relative des événements de crue inhabituels à la valeur globale d'un critère quadratique Résumé Les critères quadratiques (c'est-à-dire basés sur les carrés des résidus du modèle) sont largement utilisés pour évaluer les performances des modèles hydrologiques. Cependant, les plus fortes erreurs ont une influence assez prononcée sur les valeurs finales du critère, ce qui peut être considéré comme un inconvénient pour une évaluation complète. Cet article étudie le cas de modèles utilisés avec mise-à-jour pour la prévision en temps réel. Nous montrons que la fraction de la série de données qui conditionne effectivement le critère final est faible sur beaucoup de bassins et correspond aux pas de temps caractérisés par les plus grandes variations de débit. En fait, la mise-à-jour du modèle rend la distribution des erreurs plus pointue, donnant ainsi une importance relative encore plus grande aux pas de temps présentant les plus fortes erreurs. Par conséquent, évaluer les performances d'un modèle mis-à-jour, en utilisant un critère quadratique, renforce le fait que ces critères se concentrent davantage sur les pas de temps les plus difficiles à modéliser (qui sont les plus intéressants dans le cas de la prévision de crue à court terme).Mots clefs évaluation de modèle; critère de qualité; critère quadratique; prévision de crue; indice de persistance; hétéroscédasticité