A New Approach to Solving Stochastic Optimal Control Problems

Abstract

A conventional approach to solving stochastic optimal control problems with time-dependent uncertainties involves the use of the stochastic maximum principle (SMP) technique. For large-scale problems, however, such an algorithm frequently leads to convergence complexities when solving the two-point boundary value problem resulting from the optimality conditions. An alternative approach consists of using continuous random variables to capture uncertainty through sampling-based methods embedded within an optimization strategy for the decision variables; such a technique may also fail due to the computational intensity involved in excessive model calculations for evaluating the objective function and its derivatives for each sample. This paper presents a new approach to solving stochastic optimal control problems with time-dependent uncertainties based on BONUS (Better Optimization algorithm for Nonlinear Uncertain Systems). The BONUS has been used successfully for non-linear programming problems with static uncertainties, but we show here that its scope can be extended to the case of optimal control problems with time-dependent uncertainties. A batch reactor for biodiesel production was used as a case study to illustrate the proposed approach. Results for a maximum profit problem indicate that the optimal objective function and the optimal profiles were better than those obtained by the maximum principle.

Optimal control and the Pontryagin's principle in chemical engineering: History, theory, and challenges

“…To partially circumvent these issues, an approach that requires sampling combined with a reweighted scheme using Kernel Density Estimation was proposed in the literature. 178 This method has been applied to complex and large-scale applications such as the OC and operations management of a biodiesel batch reactor, 179 and minimization of utility consumption in power plants. 180,181 are able to obtain the optimal (or suboptimal) control actions within the sampling times.…”

“…To partially circumvent these issues, an approach that requires sampling combined with a reweighted scheme using Kernel Density Estimation was proposed in the literature 178 . This method has been applied to complex and large‐scale applications such as the OC and operations management of a biodiesel batch reactor, 179 and minimization of utility consumption in power plants 180,181 . Despite these advances in the field, more research involving different approaches or methods that can handle uncertainty more efficiently are needed to further advance the application of OC formulations in chemical engineering.…”

A perspective on the role of uncertainty in sustainability science and engineering

“…These concepts are derived from the financial and economics literature and engineering optimal control theory. These concepts could be extended to studying sustainability as presented in the recent literature ( Shastri and Diwekar, 2008 ; Shastri et al., 2008a , Shastri et al., 2008b ; Diwekar and Shastri, 2010 ; Diwekar, 2012b , Diwekar, 2015 ; Doshi et al., 2015 , Rodriguez-Gonzalez et al., 2019 ).…”

Stochastic Modelling and Stability Analysis of Large-Scale Wind Power Generation System with Dynamic Loads

“…Due to the large-scale nature of the renewable power grid being considered, continuous random variable is used which can be easily handled analytically as compared to discrete random variables. Again, for large-scale modelling, continuous random variables are preferred [11,14].…”