Input Variable Selection (IVS) is an essential step in data-driven modelling and is particularly relevant in environmental applications, where potential input variables are often collinear and redundant. While methods for IVS continue to emerge, each has its own advantages and limitations and no single method is best suited to all datasets and modelling purposes. Rigorous evaluation of IVS methods would allow their effectiveness to be properly identified in various circumstances. However, such evaluations are largely neglected due to the lack of guidelines to facilitate consistent and standardised assessment. This work proposes a new evaluation framework, which consists of benchmark datasets with the typical properties of environmental data, a recommended set of evaluation criteria and a website for sharing data and code. The framework is demonstrated on four IVS algorithms commonly used in environmental modelling studies. The results indicate interesting differences in the algorithms' performance that have not been identified previously.Response to Reviewers: Editor I have now received reviews of the above paper and these lead me to recommend that revision according to all the reviewers' comments is necessary. I may not send it back to reviewers, trusting that you will cut it down, otherwise few people will not bother reading it.Response to Editor comment No. 1. We significantly reduced the manuscript length by mostly focusing on Section 2 and 3, as also suggested by reviewer #2. Where possible, we also tried to reduce Section 5. Overall, we obtained a reduction of about 6 pages (from the introduction to the conclusion) with respect to the previous version of the manuscript. Furthermore, we removed Appendix A, since this material can be directly accessed from the framework website. This gives an overall reduction of 21 pages.Another issue is that I'd like it to fit better with EMS being a generic journal and so link to our key outputs. Most citations to EMS papers are to the authors themselves! Just one way to do this is to link with/refer to other key modelling concepts and issues in the journal. For example see the next paragraph.On model evaluation: that it is credible and addressed well. In this connection, I would like you to justify, and if pertinent expand or comment upon, your choice of evaluation metrics and methods among the ones, for example, in the recent EMS Position paper of Bennett et al (2013) on performance evaluation (they propose a 5-step procedure for evaluating the performance of models). You could also add/comment on visual methods and quantitative measures used to examine model quantities and residuals, including visual inspection. There are several other evaluation issues you could address/compare as well and the paper by Robson and cited below presents an excellent example in Section 13 of that paper. One of our aims for EMS is to strengthen the credibility and relevance of the modelling reported and do this whatever the environmental problem sector. That way your paper is mor...

Abstract. Considerable research effort has recently been directed at improving and operationalising ensemble seasonal streamflow forecasts. Whilst this creates new opportunities for improving the performance of water resources systems, there may also be associated risks. Here, we explore these potential risks by examining the sensitivity of forecast value (improvement in system performance brought about by adopting forecasts) to changes in the forecast skill for a range of hypothetical reservoir designs with contrasting operating objectives. Forecast-informed operations are simulated using rolling horizon, adaptive control and then benchmarked against optimised control rules to assess performance improvements. Results show that there exists a strong relationship between forecast skill and value for systems operated to maintain a target water level. But this relationship breaks down when the reservoir is operated to satisfy a target demand for water; good forecast accuracy does not necessarily translate into performance improvement. We show that the primary cause of this behaviour is the buffering role played by storage in water supply reservoirs, which renders the forecast superfluous for long periods of the operation. System performance depends primarily on forecast accuracy when critical decisions are made -namely during severe drought. As it is not possible to know in advance if a forecast will perform well at such moments, we advocate measuring the consistency of forecast performance, through bootstrap resampling, to indicate potential usefulness in storage operations. Our results highlight the need for sensitivity assessment in value-of-forecast studies involving reservoirs with supply objectives.

[1] Although being one of the most popular and extensively studied approaches to design water reservoir operations, Stochastic Dynamic Programming is plagued by a dual curse that makes it unsuitable to cope with large water systems: the computational requirement grows exponentially with the number of state variables considered (curse of dimensionality) and an explicit model must be available to describe every system transition and the associated rewards/costs (curse of modeling). A variety of simplifications and approximations have been devised in the past, which, in many cases, make the resulting operating policies inefficient and of scarce relevance in practical contexts. In this paper, a reinforcement-learning approach, called fitted Q-iteration, is presented: it combines the principle of continuous approximation of the value functions with a process of learning off-line from experience to design daily, cyclostationary operating policies. The continuous approximation, performed via tree-based regression, makes it possible to mitigate the curse of dimensionality by adopting a very coarse discretization grid with respect to the dense grid required to design an equally performing policy via Stochastic Dynamic Programming. The learning experience, in the form of a data set generated combining historical observations and model simulations, allows us to overcome the curse of modeling. Lake Como water system (Italy) is used as study site to infer general guidelines on the appropriate setting for the algorithm parameters and to demonstrate the advantages of the approach in terms of accuracy and computational effectiveness compared to traditional Stochastic Dynamic Programming.Citation: Castelletti, A., S. Galelli, M. Restelli, and R. Soncini-Sessa (2010), Tree-based reinforcement learning for optimal water reservoir operation, Water Resour. Res., 46, W09507,

No abstract

No abstract

Abstract. During the past decades, the increased impact of anthropogenic interventions on river basins has prompted hydrologists to develop various approaches for representing human–water interactions in large-scale hydrological and land surface models. The simulation of water reservoir storage and operations has received particular attention, owing to the ubiquitous presence of dams. Yet, little is known about (1) the effect of the representation of water reservoirs on the parameterization of hydrological models, and, therefore, (2) the risks associated with potential flaws in the calibration process. To fill in this gap, we contribute a computational framework based on the Variable Infiltration Capacity (VIC) model and a multi-objective evolutionary algorithm, which we use to calibrate VIC's parameters. An important feature of our framework is a novel variant of VIC's routing model that allows us to simulate the storage dynamics of water reservoirs. Using the upper Mekong river basin as a case study, we calibrate two instances of VIC – with and without reservoirs. We show that both model instances have the same accuracy in reproducing daily discharges (over the period 1996–2005), a result attained by the model without reservoirs by adopting a parameterization that compensates for the absence of these infrastructures. The first implication of this flawed parameter estimation stands in a poor representation of key hydrological processes, such as surface runoff, infiltration, and baseflow. To further demonstrate the risks associated with the use of such a model, we carry out a climate change impact assessment (for the period 2050–2060), for which we use precipitation and temperature data retrieved from five global circulation models (GCMs) and two Representative Concentration Pathways (RCPs 4.5 and 8.5). Results show that the two model instances (with and without reservoirs) provide different projections of the minimum, maximum, and average monthly discharges. These results are consistent across both RCPs. Overall, our study reinforces the message about the correct representation of human–water interactions in large-scale hydrological models.

No abstract

No abstract