Martingale methods in stochastic control

Abstract

The martingale treatment of stochastic control problems is based on the idea that the correct formulation of Bellman's "principle of optiraality" for stochastic minimization problems is in terms of a submartingale inequality: the "value function" of dynamic programming is always a submartingale and is a martingale under a particular control strategy if and only if that strategy is optimal. Local conditions for optimality in the form of a minimum principle can be obtained by applying Meyer's submartingale decomposition along with martingale representation theorems; conditions for existence of an optimal strategy can also be stated. This paper gives an introduction to these methods and a survey of the results that have been obtained so far, as well as an indication of some shortcomings in the theory and open problems. By way of introduction we treat systems of controlled stochastic differential equations, the case for which the most definitive results have been obtained so far. We then outline a general semimartingale formulation of controlled processes, state some optimality conditions and indicate their application to other specific cases such as that of controlled jump processes.The martingale approach to some related problems -optimal stopping, impulse control and stochastic differential games -will also be outlined.

Hedging with small uncertainty aversion

“…Consider the Black-Scholes-Barenblatt equation 17) where ψ > 0 is a (small) parameter and λ ≤ 0 ≤ Λ are suitable functions. This equation corresponds to the problem of finding the smallest initial capital that allows to superreplicate the option G(S T ) for any volatility process evolving in the random interval…”

“…The central idea is to find an asymptotic solution to the Hamilton-Jacobi-BellmanIsaacs (henceforth HJBI) equation associated to the hedging problem and corresponding almost optimal controls. For the convenience of the reader, we provide a derivation of the HJBI equation which starts from a general sufficient criterion for optimality (Proposition 4.1) known as principle of optimality [17] or martingale optimality principle [60, V.15]; see also [61,Proposition 4.1] for a version of the martingale optimality principle in the context of a zero-sum game. After that, we explain how the HJBI equation together with an appropriate ansatz can be used to derive candidates for the value and the almost optimal controls of our hedging problems.…”

“…Here, θ runs through some subset of trading strategies such that · 0 (θ t −V s (t, S t )) dS t is a supermartingale (to rule out doubling strategies) and U is a (strictly concave) utility function on R. As the option can be perfectly replicated, this is a trivial example of a utility maximisation problem 17 This error incurred from hedging with a misspecified volatility is well known in the literature and in practice. To the best of our knowledge, it appeared first in Lyons [48,Equation (27)] (see also [47, Equation (6.7)]).…”

Stochastic Optimal Control

“…Following Bellman, several works focus on finding conditions under which HJB equations have solutions (see survey in [14][15][16][17][18]). Establishing such conditions often limits the class of problems that can be handled by the dynamic programming approach [19].…”

Controle de sistemas não-Markovianos

“…Devido a grande dificuldade de tratar o caso não-Markoviano, as primeiras caracterizações iniciarão-se com o controle apenas no drift (função α da EDE (11), (16), (19)) e não no coeficiente de difusão (função σ da EDE).…”