Deep Neural Networks Algorithms for Stochastic Control Problems on Finite Horizon: Convergence Analysis

Huré, Côme; Pham, Huyên; Bachouch, Achref; Langrené, Nicolas

doi:10.1137/20m1316640

Cited by 56 publications

(31 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are two key benefits of doing this. Firstly, it reconciles the potential errors of Han and W. (2016); Huré et al (2021) where the algorithm gets stuck at a local optimiser. To find the value function (given a control) we ensure that the terminal condition of the HJB equation holds, which involves minimising the least squares error of the associated processes at the terminal time.…”

Section: Introductionmentioning

confidence: 82%

“…This is not explicitly given a name by the authors, so we refer to it as the PVO (Primal Value Optimisation) algorithm. Secondly we compare to the Hybrid-Now algorithm of Huré et al (2021) (referred to in the sequel as Hybrid), where the control is optimised using an approximation of the value function, which is also found using neural networks. First, we note that while both algorithms search for a Markovian control, and as such solve Markovian control problems, they are potentially less restrictive than our algorithm.…”

Section: Comparison To Literaturementioning

confidence: 99%

“…However, the computation costs may be high, as one introduces a discretisation with many time steps or a high dimensional control space, and it only outputs the value at time 0. Huré et al (2021) finds a local approximation of the value function with machine learning. However, it only holds for Markovian problems and the iterative nature of this value function approximation may lead to error propagation.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Deep Learning for Constrained Utility Maximisation

Davey

Zheng

2021

Methodol Comput Appl Probab

View full text Add to dashboard Cite

This paper proposes two algorithms for solving stochastic control problems with deep learning, with a focus on the utility maximisation problem. The first algorithm solves Markovian problems via the Hamilton Jacobi Bellman (HJB) equation. We solve this highly nonlinear partial differential equation (PDE) with a second order backward stochastic differential equation (2BSDE) formulation. The convex structure of the problem allows us to describe a dual problem that can either verify the original primal approach or bypass some of the complexity. The second algorithm utilises the full power of the duality method to solve non-Markovian problems, which are often beyond the scope of stochastic control solvers in the existing literature. We solve an adjoint BSDE that satisfies the dual optimality conditions. We apply these algorithms to problems with power, log and non-HARA utilities in the Black-Scholes, the Heston stochastic volatility, and path dependent volatility models. Numerical experiments show highly accurate results with low computational cost, supporting our proposed algorithms.

show abstract

Section: Introductionmentioning

confidence: 82%

Section: Comparison To Literaturementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning for Constrained Utility Maximisation

Davey

Zheng

2021

Methodol Comput Appl Probab

View full text Add to dashboard Cite

show abstract

“…Finally, it is worth mentioning that machine and deep learning techniques have proven to provide effective algorithms able to efficiently solve several stochastic optimal control problems; see, e.g., Bachouch et al (2018); Deschatre and Mikael (2020); Huré et al (2018). Therefore, our future investigations will be extensively carried out so as to consider the impulse type optimal control tasks proposed here while exploiting innovative neural networks solutions.…”

Section: Discussionmentioning

confidence: 99%

A Bank Salvage Model by Impulse Stochastic Controls

Cordoni

Persio

Jiang

2020

Risks

View full text Add to dashboard Cite

The present paper is devoted to the study of a bank salvage model with a finite time horizon that is subjected to stochastic impulse controls. In our model, the bank’s default time is a completely inaccessible random quantity generating its own filtration, then reflecting the unpredictability of the event itself. In this framework the main goal is to minimize the total cost of the central controller, which can inject capitals to save the bank from default. We address the latter task, showing that the corresponding quasi-variational inequality (QVI) admits a unique viscosity solution—Lipschitz continuous in space and Hölder continuous in time. Furthermore, under mild assumptions on the dynamics the smooth-fit W l o c ( 1 , 2 ) , p property is achieved for any 1 < p < + ∞ .

show abstract

“…In [2,80] the authors introduce and compare a number of neural network‐based algorithms applied to stochastic control problems, nonlinear PDEs and BSDEs, incl. the example discussed in Section 4.3.…”

Section: Extensions and Related Workmentioning

confidence: 99%

Three ways to solve partial differential equations with neural networks — A review

2021

View full text Add to dashboard Cite

Neural networks are increasingly used to construct numerical solution methods for partial differential equations. In this expository review, we introduce and contrast three important recent approaches attractive in their simplicity and their suitability for high‐dimensional problems: physics‐informed neural networks, methods based on the Feynman–Kac formula and methods based on the solution of backward stochastic differential equations. The article is accompanied by a suite of expository software in the form of Jupyter notebooks in which each basic methodology is explained step by step, allowing for a quick assimilation and experimentation. An extensive bibliography summarizes the state of the art.

show abstract

Deep Neural Networks Algorithms for Stochastic Control Problems on Finite Horizon: Convergence Analysis

Cited by 56 publications

References 8 publications

Deep Learning for Constrained Utility Maximisation

Deep Learning for Constrained Utility Maximisation

A Bank Salvage Model by Impulse Stochastic Controls

Three ways to solve partial differential equations with neural networks — A review

Contact Info

Product

Resources

About