A survey of numerical solutions for stochastic control problems: Some recent progress

Abstract

<p style='text-indent:20px;'>This paper presents a survey on some of the recent progress on numerical solutions for controlled switching diffusions. We begin by recalling the basics of switching diffusions and controlled switching diffusions. We then present regular controls and singular controls. The main objective of this paper is to provide a survey on some recent advances on Markov chain approximation methods for solving stochastic control problems numerically. A number of applications in insurance, mathematical biology, epidemiology, and economics are presented. Several numerical examples are provided for demonstration.</p>

Razumikhin technique for stabilisation of highly nonlinear hybrid systems by bounded discrete-time state feedback control working intermittently

“…Design the control function to meet Rule 2.6. Then for any initial data x 0 and r 0 , there exists a unique global solution x(t) of the controlled SDE (7). Moreover it satisfies that for any…”

“…Let all the conditions in Theorem 2.9 hold. Then the solution of the controlled SDE (7) satisfies that for any t ∈ R + E Ū (x(t), x(t τ ), t, r(t))…”

“…However, the observation duration τ obtained by using this method is always awesome. Even worse, it only works well when the underlying hybrid SDEs are globally Lipschitz continuous (see Appendix in [5]), which might exclude many important practical systems such as stochastic volatility model [4,8], biological system [24,7], and oscillator network [12]. It is hence necessary to develop new techniques to deal with this stabilisation problem to improve the value of τ and cover more general models.…”

“…If not, we would like to design a state feedback control based on discrete-time observations working intermittently, u(x(t τ ), t, r(t))I(t), to make the controlled SDE dx(t) = f (x(t), t, r(t)) + u(x(t τ ), t, r(t))I(t) dt + g(x(t), t, r(t))dW (t) (7) become stable.…”

Recent Developments in Machine Learning Methods for Stochastic Control and Games

“…Besides the methods reviewed in this paper, there is a rich literature on methods, some of them related to machine learning but without neural networks. For example, to cite just a few examples, Markov chain based methods [38], regression based methods [18,34], and approaches based on PDEs and BSDEs [68,89]; see, e.g., the survey papers [165,153] and the references therein. Furthermore, we focus mostly on standard classes of stochastic control problems and games but many other problems are considered in the literature.…”