Using stochastic dual dynamic programming in problems with multiple near‐optimal solutions

Rougé, Charles; Tilmant, Amaury

doi:10.1002/2016wr018608

Cited by 22 publications

(14 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, stochastic dynamic programming generally allows more decision epochs and stochastic variables due to the exponential increase in the number of scenarios for multistage stochastic programming with these factors. Stochastic dual dynamic programming aims to combine the advantages of these methods by decomposing the problem into a number of subproblems, each of which focuses on decisions at a single epoch and approximates the expected future profit by a series of affine constraints or "cuts" [177]. However, this method is most effective for problems that can be represented as linear models.…”

Section: Type Of Mathematical Programmingmentioning

confidence: 99%

Review of Mathematical Programming Applications in Water Resource Management Under Uncertainty

Archibald

Marshall

2018

Environ Model Assess

View full text Add to dashboard Cite

Economic development, variation in weather patterns and natural disasters focus attention on the management of water resources. This paper reviews the literature on the development of mathematical programming models for water resource management under uncertainty between 2010 and 2017. A systematic search of the academic literature identified 448 journal articles on water resource management for examination. Bibliometric analysis is employed to investigate the methods that researchers are currently using to address this problem and to identify recent trends in research in the area. The research reveals that stochastic dynamic programming and multistage stochastic programming are the methods most commonly applied. Water resource allocation, climate change, water quality and agricultural irrigation are amongst the most frequently discussed topics in the literature. A more detailed examination of the literature on each of these topics is included. The findings suggest that there is a need for mathematical programming models of large-scale water systems that deal with uncertainty and multiobjectives in an effective and computationally efficient way.

show abstract

Section: Type Of Mathematical Programmingmentioning

confidence: 99%

Review of Mathematical Programming Applications in Water Resource Management Under Uncertainty

Archibald

Marshall

2018

Environ Model Assess

View full text Add to dashboard Cite

show abstract

“…Stochastic dual dynamic programming (SDDP; Pereira, ) is one of the few solutions available for the optimization of large‐scale reservoir systems and the production of sensible operating policies. It can be used even in situations where data availability on the system itself is limited (Rougé & Tilmant, ). Section 2.4 provides details on this methodology.…”

Section: Framework: Building a Best Casementioning

confidence: 99%

Identifying Key Water Resource Vulnerabilities in Data‐Scarce Transboundary River Basins

Rougé

Tilmant

Zaitchik

et al. 2018

Water Resources Research

Self Cite

View full text Add to dashboard Cite

This paper presents a two‐step framework to identify key water resource vulnerabilities in transboundary river basins where data availability on both hydrological fluxes and the operation of man‐made facilities is either limited or nonexistent. In a first step, it combines two state‐of‐the‐art modeling tools to overcome data limitations and build a model that provides a lower bound on risks estimated in that basin. Land data assimilation (process‐based hydrological modeling taking remote‐sensed products as inputs) is needed to evaluate hydrological fluxes, that is, streamflow data and consumptive use in irrigated agriculture—a lower‐end estimate of demand. Hydroeconomic modeling provides cooperative water allocation policies that reflect the best‐case management of storage capacity under hydrological uncertainty at a monthly time step for competing uses—hydropower, irrigation. In a second step, the framework uses additional scenarios to proceed with the in‐depth analysis of the vulnerabilities identified despite the use of what is by definition a best‐case model. We implement this approach to the Tigris‐Euphrates river basin, a politically unstable region where water scarcity has been hypothesized to serve as a trigger for the Syrian revolution and ensuing war. Results suggest that even under the framework's best‐case assumptions, the Euphrates part of the basin is close to a threshold where it becomes reliant on transfers of saline water from other parts of the basin to ensure irrigation demands are met. This Tigris‐Euphrates river basin application demonstrates how the proposed framework quantifies vulnerabilities that have been hitherto discussed in a mostly qualitative, speculative way.

show abstract

“…In the explicit stochastic optimization (ESO), probability distributions of random variables are used to derive a transition probability matrix (TPM) required for modeling dynamics of the underlying stochastic process. Several stochastic optimization methods have been applied to optimal multi-reservoir operations problems such as chance-constrained programming [12,13], reliability programming [14][15][16], the Fletcher-Ponnambalam (FP) method that does not require discretization or inflow scenarios [17][18][19], stochastic LP [20], stochastic dynamic programming (SDP), with some approximations for dimensionality reduction [21][22][23][24][25][26][27][28][29][30][31][32][33][34][35], stochastic dual DP (SDDP) [36][37][38][39][40], sampling SDP (SSDP) [41], and reinforcement learning (RL) [6,42,43].…”

Section: Introductionmentioning

confidence: 99%

Aggregation–Decomposition-Based Multi-Agent Reinforcement Learning for Multi-Reservoir Operations Optimization

Hooshyar

Mousavi

Mahootchi

et al. 2020

Water

View full text Add to dashboard Cite

Stochastic dynamic programming (SDP) is a widely-used method for reservoir operations optimization under uncertainty but suffers from the dual curses of dimensionality and modeling. Reinforcement learning (RL), a simulation-based stochastic optimization approach, can nullify the curse of modeling that arises from the need for calculating a very large transition probability matrix. RL mitigates the curse of the dimensionality problem, but cannot solve it completely as it remains computationally intensive in complex multi-reservoir systems. This paper presents a multi-agent RL approach combined with an aggregation/decomposition (AD-RL) method for reducing the curse of dimensionality in multi-reservoir operation optimization problems. In this model, each reservoir is individually managed by a specific operator (agent) while co-operating with other agents systematically on finding a near-optimal operating policy for the whole system. Each agent makes a decision (release) based on its current state and the feedback it receives from the states of all upstream and downstream reservoirs. The method, along with an efficient artificial neural network-based robust procedure for the task of tuning Q-learning parameters, has been applied to a real-world five-reservoir problem, i.e., the Parambikulam–Aliyar Project (PAP) in India. We demonstrate that the proposed AD-RL approach helps to derive operating policies that are better than or comparable with the policies obtained by other stochastic optimization methods with less computational burden.

show abstract

Using stochastic dual dynamic programming in problems with multiple near‐optimal solutions

Cited by 22 publications

References 37 publications

Review of Mathematical Programming Applications in Water Resource Management Under Uncertainty

Review of Mathematical Programming Applications in Water Resource Management Under Uncertainty

Identifying Key Water Resource Vulnerabilities in Data‐Scarce Transboundary River Basins

Aggregation–Decomposition-Based Multi-Agent Reinforcement Learning for Multi-Reservoir Operations Optimization

Contact Info

Product

Resources

About