Optimizing Trading Decisions for Hydro Storage Systems Using Approximate Dual Dynamic Programming

Abstract

We propose a new approach to optimize operations of hydro storage systems with multiple connected reservoirs which participate in wholesale electricity markets. Our formulation integrates short-term intraday with long-term interday decisions. The intraday problem considers bidding decisions as well as storage operation during the day and is formulated as a stochastic program. The interday problem is modeled as a Markov decision process of managing storage operation over time, for which we propose integrating stochastic dual dynamic programming with approximate dynamic programming. We show that the approximate solution converges towards an upper bound of the optimal solution. To demonstrate the efficiency of the solution approach, we fit an econometric model to actual price and inflow data and apply the approach to a case study of an existing hydro storage system. Our results indicate that the approach is tractable for a real-world application and that the gap between theoretical upper and a simulated lower bound decreases sufficiently fast.

A comparison of approximate dynamic programming techniques on benchmark energy storage problems: Does anything work?

“…The mathematical formulation is similar regardless of the exact application; [7] and [8], for example, present different techniques (including optimal switching and ADP) to study control policies of natural gas storage facilities. Moreover, [9] and [10] study the optimization of a hydroelectric reservoir, with the additional complication of bidding day-ahead. The second paper, [10], uses a method based on stochastic dual dynamic programming (SDDP).…”

“…Moreover, [9] and [10] study the optimization of a hydroelectric reservoir, with the additional complication of bidding day-ahead. The second paper, [10], uses a method based on stochastic dual dynamic programming (SDDP). SDDP and its related methods use Benders cuts, but the theoretical work in this area uses the assumption that random variables only have a finite set of outcomes [11] (and thus difficult to scale to larger problems).…”

Sell or store? An ADP approach to marketing renewable energy

“…Hannah and Dunson (2011) apply ADP to the day-ahead commitment problem of a wind farm in combination with energy storage. Löhndorf et al (2013) study the optimization of hydro-storage systems when participating in both the intraday and day-ahead markets. Sioshansi et al (2014) determine the capacity value of energy storage by means of dynamic programming.…”

Stochastic Optimization of Grid to Vehicle Frequency Regulation Capacity Bids

“…The stochastic dual dynamic programming (SDDP) method of [16] can be thought of as a specific implementation of nested Benders decomposition, which avoids the discretization of controlled endogenous state variables. Reference [17] and more recently [18] extend [16] and propose solution methods which consider uncertain and Markovian energy prices. These methods approximate the exogenous random process for price with a finite set of states and transition probabilities, and are described as a combination of SDP and SDDP.…”

“…However, the methods proposed in [16]- [18] cannot be directly applied to the G2V problem. These methods do not allow uncertain decision variable coefficients in the state transition equation.…”