We propose a Hilbert-valued perturbed subgradient algorithm with stochastic noises, and we provide a convergence proof for this algorithm under classical assumptions on the descent direction and new assumptions on the stochastic noises. Instead of requiring the stochastic noises to correspond to martingale increments, we only require these noises to be asymptotically so. Furthermore, the variance of these noises is allowed to grow infinitely under the control of a decreasing sequence linked with the subgradient stepsizes. This algorithm can be used to solve stochastic closed-loop control problems without any a priori discretization of the uncertainty, such as linear decision rules or tree representations. It can also be used as a way to perform stochastic dynamic programming without state-space discretization or a priori functional bases (i.e., approximate dynamic programming). Both problems arise frequently—for example, in power systems scheduling or option pricing. This article focuses on the theorical foundations of the algorithm, and gives references to detailed practical experimentations.
In this paper, we present an Uzawa-based heuristic that is adapted to some type of stochastic optimal control problems. More precisely, we consider dynamical systems that can be divided into small-scale independent subsystems, though linked through a static almost sure coupling constraint at each time step. This type of problem is common in production/portfolio management where subsystems are, for instance, power units, and one has to supply a stochastic power demand at each time step. We outline the framework of our approach and present promising numerical results on a simplified power management problem
We focus on solving closed-loop stochastic problems, and propose a perturbed gradient algorithm to achieve this goal. The main hurdle in such problems is the fact that the control variables are infinite dimensional, and have hence to be represented in a finite way in order to numerically solve the problem. In the same way, the gradient of the criterion is itself an infinite dimensional object. Our algorithm replaces this exact (and unknown) gradient by a perturbed one, which consists in the product of the true gradient evaluated at a random point and a kernel function which extends this gradient to the neighbourhood of the random point. Proceeding this way, we explore the whole space iteration after iteration through random points. Since each kernel function is perfectly known by a finite (and small) number of parameters, say N , the control at iteration k is perfectly known as an infinite dimensional object by at most N × k parameters. The main strength of this method is that it avoids any discretization of the underlying space, provided that we can draw as many points as needed in this space. Hence, we can take into account in a new way the possible measurability constraints of the problem. Moreover, the randomization of this algorithm implies that the most probable parts of the space are the most explored ones, what is a priori an interesting feature.In this paper, we first show a convergence result of this algorithm in the general case, and then give a few numerical examples showing the interest of this method for solving practical stochastic optimization problems.
International audienceNuclear power plants must be regularly shut down in order to perform refueling and maintenance operations. The scheduling of the outages is the first problem to be solved in electricity production management. It is a hard combinatorial problem for which an exact solving is impossible. Our approach consists in modelling the problem by a two-level problem. First, we fix a feasible schedule of the dates of the outages. Then, we solve a low-level problem of optimization of elecricity production, by respecting the initial planning. In our model, the low-level problem is a deterministic convex optimal control problem. Given the set of solutions and Lagrange multipliers of the low-level problem, we can perform a sensitivity analysis with respect to dates of the outages. The approximation of the value function which is obtained could be used for the optimization of the schedule with a local search algorithm.Les centrales nucléaires doivent être régulièrement arrêtées afin de réaliser des opérations de maintenance et de rechargement en combustible nucléaire. La planification de ces arrêts constitue le premier problème à résoudre en gestion de la production d'électricité. C'est un problème combinatoire difficile qui ne peut être résolu exactement. Notre approche consiste à modéliser ce problème par un problème à deux niveaux. Tout d'abord, nous fixons un calendrier admissible des dates des arrêts des centrales. Puis, nous résolvons un sous-problème de production d'électricité, en respectant le calendrier initial. Dans notre modèle, ce sous-problème est un problème de contrôle optimal déterministe et convexe. Etant donnés les solutions et multiplicateurs de Lagrange du sous-problème, nous pouvons réaliser une analyse de sensibilité par rapport aux dates des arrêts. L'approximation de la fonction valeur que nous obtenons devrait permettre de mettre en place un algorithme de recherche locale pour l'optimisation de ces dates d'arrêts
We present an algorithm for American option pricing based on stochastic approximation techniques. Besides working on a finite subset of the exercise dates (e.g. considering the associated Bermudean option), option pricing algorithms generally involve another step of discretization, either on the state space or on the underlying functional space. Our work, which is an application of a more general perturbed gradient algorithm introduced recently by the authors, consists in approximating the value functions of the classical dynamic programming equation at each time step by a linear combination of kernels. The so-called kernel-based stochastic gradient algorithm avoids any a priori discretization, besides the discretization of time. Thus, it converges toward the optimum of the non-discretized Bermudan option pricing problem. We present a comprehensive methodology to implement efficiently this algorithm, including discussions on the numerical tools used, like the Fast Gauss Transform, or Brownian bridge. We also compare our results to some existing methods, and provide empirical statistical results.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.