Inferring efficient operating rules in multireservoir water resource systems: A review

Most water reservoir operators make use of forecasts to inform their decisions and enhance water systems flexibility and resilience by anticipating hydrological extremes. Yet, despite numerous candidate hydro‐meteorological variables and forecast horizons may potentially be beneficial to operations, the best information set for a given problem is often not evident. Additionally, in multipurpose systems characterized by multiple demands with varying vulnerabilities and temporal scales, this information set might change according to the objective tradeoff. In this work, we contribute a novel method to learn the optimal policy representation (i.e., policy input set) by combining a feature selection routine with a multiobjective Direct Policy Search framework in order to retrieve the best policy input set online (i.e., while learning the policy) and dynamically with the objective trade‐off. The selected policy search routine is the Neuro‐Evolutionary Multi‐Objective Direct Policy Search (NEMODPS) which generates flexible policy shapes adaptive to online changes in the input set. This approach is demonstrated on the case study of Lake Como (Italy), where the operating objectives are highly heterogeneous in their dynamics (fast and slow) and vulnerabilities (wet and dry extremes). We show how varying objectives, and tradeoffs therein, benefit from a different policy representation, ultimately yielding remarkable results in terms of conflict mitigation between different users. More informed policies, moreover, show higher robustness when re‐evaluated across a suite of different hydrological conditions.

“…Following Giuliani et al. (2015), the Perfect Operating Policy π POP is designed by solving Problem 1 under the hypothesis of deterministic knowledge of the trajectory

{\varepsilon }_{1}^{H}

of external drivers over the entire evaluation horizon H at any given time step,

{\pi }^{POP}={\pi }^{POP}({s}_{t},t,{\varepsilon }_{1}^{H})

Section: Methodsmentioning

confidence: 99%

Policy Representation Learning for Multiobjective Reservoir Policy Design With Different Objective Dynamics

Zaniolo

Giuliani

Castelletti

2021

“…Water resource systems researchers have developed a broad range of strategies for dynamically managing reservoir operations in the face of uncertain hydrometeorology and demands (see reviews by Castelletti et al., 2008; Labadie, 2004; Macian‐Sorribes & Pulido‐Velazquez, 2019; Yeh, 1985), but Stochastic Dynamic Programming (SDP) and its many derivatives have been the most popular. The problem is formulated as an MDP in which a decision‐maker must make sequential decisions based on the stochastically evolving state of the system.…”

Section: Introductionmentioning

confidence: 99%

From Stream Flows to Cash Flows: Leveraging Evolutionary Multi‐Objective Direct Policy Search to Manage Hydrologic Financial Risks

Hamilton

Characklis

Reed

2022

Reservoir control and financial risk management share strong similarities. The principal task in each is to reduce the risk of negative impacts from variable inflows (either hydrologic flows or cash flows), through the use of a buffer stock (either a reservoir or a reserve fund) that is filled in times of abundance and drawn down in times of scarcity (Figure 1). Other risk management tools may also be used to limit the impact of low-flow periods, but at a cost (e.g., water desalination or demand management for streamflow deficits, and borrowing or financial hedging for cash flow deficits). In both cases, the manager must make decisions under an array of uncertainties, and may need to navigate tradeoffs between conflicting objectives (e.g., flood control vs. water supply for reservoir control, risk vs. cost for financial risk management). And in both cases, as systems dynamically evolve, managers will have to adapt to new information as it becomes available. In other words, reservoir control and financial risk management can be formulated as very similar Markov Decision Processes (MDPs) (Bertsekas, 2019;Powell, 2019), whether managers attempt to solve this problem explicitly, using programmatic approaches such as stochastic dynamic programming, or implicitly, relying on expert specified rules. Additionally, reservoir control and financial risk management are strongly interdependent activities for water-reliant organizations in the Food-Energy-Water Nexus, such as hydropower producers, municipal water utilities, and irrigation districts (Cai et al., 2018;D'Odorico et al., 2018;Scanlon et al., 2017). Such organizations rely on water for the provision of services, and as a result, their revenues and/or costs can be highly dependent on hydrologic inflows (Blomfield & Plummer, 2014;Larson et al., 2012). This suggests that an understanding of complex water resource system dynamics can be used to better characterize and adaptively manage financial risks borne by water-reliant organizations.

“…The problem of designing optimal reservoir operations has been extensively studied since the seminal works by Rippl (1883), Hazen (1914), and Varlet (1923). Over the years, several review papers (e.g., Dobson et al., 2019; Klemeš, 1987; Labadie, 2004; Macian‐Sorribes & Pulido‐Velazquez, 2020; Simonovic, 1992; Stedinger et al., 2013; Wurbs, 1993; Yakowitz, 1982; Yeh, 1985) have described the evolving state of the art in the field. Changes in societal perceptions of natural resources and increasing environmental awareness are modifying and enlarging the number of objectives considered (e.g., Schmitt et al., 2018; Wild et al., 2019; Winemiller et al., 2016).…”

Section: Introductionmentioning

confidence: 99%

“…In particular, we provide a critical analysis of the major emerging challenges for optimal reservoir control methods that support the design of closed‐loop policies, as derived from the 115 publications that use either AVS or AVP methods. We do not discuss in detail the other methods (i.e., Mathematical Programming, Open‐Loop decisions, and Rule Curves) that rely on simplified problem formulations and that have already been well covered in existing review papers (e.g., Labadie, 2004; Macian‐Sorribes & Pulido‐Velazquez, 2020).…”

Section: Introductionmentioning

confidence: 99%

A State‐of‐the‐Art Review of Optimal Reservoir Control for Managing Conflicting Demands in a Changing World

Giuliani

Lamontagne

Reed

et al. 2021

The state of the art for optimal water reservoir operations is rapidly evolving, driven by emerging societal challenges. Changing values for balancing environmental resources, multisectoral human system pressures, and more frequent climate extremes are increasing the complexity of operational decision making. Today, reservoir operations benefit from technological advances, including improved monitoring and forecasting systems as well as increasing computational power. Past research in this area has largely focused on improving solution algorithms within the limits of the available computational power, using simplified problem formulations that can misrepresent important systemic complexities and intersectoral interactions. In this study, we review the recent literature focusing on how the operation design problem is formulated, rather than solved, to address existing challenges and take advantage of new opportunities. This paper contributes a comprehensive classification of over 300 studies published over the last years into distinctive categories depending on the adopted problem formulation, which clarifies consolidated methodological approaches and emerging trends. Our analysis also suggests that control policy design methods may benefit from broadening the types of information that is used to condition operational decisions, and from using emulation modeling to identify low‐order, computationally efficient surrogate models capturing realistic representations of river basin systems' complexity in order to isolate key decision‐relevant processes. These advances in reservoir operations hold significant promise for better addressing the challenges of conflicting human pressures and a changing world, which is particularly important, given the renewed interest in dam construction globally.