Exact Converging Bounds for Stochastic Dual Dynamic Programming via Fenchel Duality

Abstract

The Stochastic Dual Dynamic Programming (SDDP) algorithm has become one of the main tools to address convex multistage stochastic optimal control problem. Recently a large amount of work has been devoted to improve the convergence speed of the algorithm through cut-selection and regularization, or to extend the field of applications to non-linear, integer or risk-averse problems. However one of the main downside of the algorithm remains the difficulty to give an upper bound of the optimal value, usually estimated through Monte Carlo methods and therefore difficult to use in the algorithm stopping criterion. In this paper we present a dual SDDP algorithm that yields a converging exact upper bound for the optimal value of the optimization problem. Incidently we show how to compute an alternative control policy based on an inner approximation of Bellman value functions instead of the outer approximation given by the standard SDDP algorithm. We illustrate the approach on an energy production problem involving zones of production and transportation links between the zones. The numerical experiments we carry out on this example show the effectiveness of the method.

The policy graph decomposition of multistage stochastic programming problems

“…This is not unique to our algorithm and is common in SDDP‐type algorithms. (Some recent work has been done on upper bounds in the linear policy graph case; see, e.g., .) In the case when all of the risk measures in the policy graph are the expectation operator, an unbiased estimate for the upper bound for the problem can be obtained by performing a Monte Carlo simulation of the policy.…”

“…The upper bound discussed above is statistical. Recently, some work has explored deterministic upper bounds , but we have not attempted to adapt those results to our setting. In lieu of a deterministic upper bound, the question of when to terminate SDDP‐type algorithms is an open question.…”

Risk-Averse Stochastic Optimal Control: an efficiently computable statistical upper bound

“…Initially described for risk-neutral linear problems, SDDP method has generated a rich literature and many variants in the past three decades. For instance we can mention: the incorporation of interstage dependent stochastic processes ( [12,22,26,33]); cut selection strategies [30,14,3]; convergence proof for risk-neutral linear problems in [31], for risk-neutral nonlinear problems in [11], for risk-averse linear and nonlinear problems in [13], and convergence proof for SDDP with cut selection in [14,3]; inexact variants (ISDDP) that approximately solve the primal and dual subproblems, first designed for linear problems and differentiable nonlinear problems in [15], then for nondifferentiable problems in [18]; the dual variants Dual SDDP from [24] and [20]; SDDP for problems with a random number of stages [16]; variants for non-convex problems: [42,21], MIDAS for problems with monotonic Bellman functions [29], problems with interstage dependent cost coefficients [20], and problems with integer variables [43]; complexity analysis [23]; StoDCuP (Stochastic Dynamic Cutting Plane) variant and its inexact variant Inexact StoDCuP introduced in [17] which linearize all or some of the nonlinear cost and constraint functions; adaptation to periodical problems with infinite horizon [39,36]; risk-averse variants.…”

“…which is a natural extension of similar constructions for two stage programs (e.g., [8, section 9.5]). Recently, two variants of Dual SDDP were introduced that also compute a deterministic upper bound, in [24] using conjugate duality and in [20] using Lagrangian duality. The bounds in [24,20] were developed for risk-neutral problems, and recently extended to risk-averse problems in [9].…”

“…Recently, two variants of Dual SDDP were introduced that also compute a deterministic upper bound, in [24] using conjugate duality and in [20] using Lagrangian duality. The bounds in [24,20] were developed for risk-neutral problems, and recently extended to risk-averse problems in [9]. However, the computational bulk required to compute the deterministic bounds from [32] and [9] for risk-averse problems increases rapidly with increase of the number of stages, the number of realizations of the stochastic data per stage, and the dimension of the state vectors.…”