We prove the almost-sure convergence of a class of sampling-based nested decomposition algorithms for multistage stochastic convex programs in which the stage costs are general convex functions of the decisions and uncertainty is modelled by a scenario tree. As special cases, our results imply the almost-sure convergence of stochastic dual dynamic programming, cutting-plane and partial-sampling (CUPPS) algorithm, and dynamic outer-approximation sampling algorithms when applied to problems with general convex cost functions.Keywords: stochastic programming; dynamic programming; stochastic dual dynamic programming algorithm; Monte-Carlo sampling; Benders decomposition MSC2000 subject classification: Primary: 90C14; secondary: 90C39 OR/MS subject classification: Primary: stochastic programming; secondary: dynamic programming Having general convex stage costs does not preclude the use of cutting plane algorithms to attack these problems. Kelley's cutting plane method (Kelley [7]) was originally devised for general convex objective functions, and can be shown to converge to an optimal solution (see, e.g., Ruszczyński [14, Theorem 7.7]), although on some instances this convergence might be very slow (Nesterov [9]). Our goal in this paper is to extend the convergence result of Ruszczyński [14] to the setting of multistage stochastic convex programming.The most well-known application of cutting planes in multistage stochastic programming is the stochastic dual dynamic programming (SDDP) algorithm of Pereira and Pinto [10]. This algorithm constructs feasible dynamic programming (DP) policies using an outer approximation of a (convex) future cost function that is computed using Benders cuts. The policies defined by these cuts can be evaluated using simulation and their performance measured against a lower bound on their expected cost. This provides a convergence criterion that may be applied to terminate the algorithm when the estimated cost of the candidate policy is close enough to its lower bound. The SDDP algorithm has led to a number of related methods (Chen and Powell [1], Donohue [2], Donohue and Birge [3], Hindsberger and Philpott [6], Philpott and Guan [11]) that are based on the same essential idea but that seek to improve the method by exploiting the structure of particular applications. We call these methods DOASA (dynamic outer-approximation sampling algorithms), but they are now generically named SDDP methods.SDDP methods are known to converge almost surely on a finite scenario tree when the stage problems are linear programs. The first formal proof of such a result was published by Chen and Powell [1], who derived this for their cutting-plane and partial-sampling (CUPPS) algorithm. This proof was extended by Linowsky and Philpott [8] to cover other SDDP algorithms. The convergence proofs in Chen and Powell [1] and Linowsky and Philpott [8] make use of an unstated assumption regarding the independence of sampled random variables and convergent subsequences of algorithm iterates. This assumption was identifi...
Abstract. For a sequence of dynamic optimization problems, we aim at discussing a notion of consistency over time. This notion can be informally introduced as follows. At the very first time step t 0 , the decision maker formulates an optimization problem that yields optimal decision rules for all the forthcoming time step t 0 , t 1 , . . . , T ; at the next time step t 1 , he is able to formulate a new optimization problem starting at time t 1 that yields a new sequence of optimal decision rules. This process can be continued until final time T is reached. A family of optimization problems formulated in this way is said to be time consistent if the optimal strategies obtained when solving the original problem remain optimal for all subsequent problems. The notion of time consistency, well-known in the field of Economics, has been recently introduced in the context of risk measures, notably by Artzner et al. (2007) and studied in the Stochastic Programming framework by Shapiro (2009) and for Markov Decision Processes (MDP) by Ruszczynski (2009). We here link this notion with the concept of "state variable" in MDP, and show that a significant class of dynamic optimization problems are dynamically consistent, provided that an adequate state variable is chosen.
In this paper, we present an Uzawa-based heuristic that is adapted to some type of stochastic optimal control problems. More precisely, we consider dynamical systems that can be divided into small-scale independent subsystems, though linked through a static almost sure coupling constraint at each time step. This type of problem is common in production/portfolio management where subsystems are, for instance, power units, and one has to supply a stochastic power demand at each time step. We outline the framework of our approach and present promising numerical results on a simplified power management problem
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.