State-Constrained Stochastic Optimal Control Problems via Reachability Approach

Abstract

Abstract. This paper deals with a class of stochastic optimal control problems (SOCP) in presence of state-constraints. It is well-known that for such problems the value function is, in general, discontinuous and its characterization by a Hamilton-Jacobi equation requires additional assumptions involving an interplay between the boundary of the set of constraints and the dynamics of the controlled system. Here, we give a characterization of the epigraph of the value function without assuming the usual controllability assumptions. For this end, the SOCP is first translated into a state-constrained stochastic target problem. Then a level-set approach is used to describe the backward reachable sets of the new target problem. It turns out that these backward-reachable sets describe the value function. The main advantage of our approach is that it allows to handle easily the state constraints by an exact penalization. However, the target problem involves a new state variable and a new control variable that is unbounded.

A Verification Theorem for Optimal Stopping Problems with Expectation Constraints

“…They extend the state space by a conditional probability process in order to reduce the problem to a standard stochastic target problem. Bokanowski, Picarelli and Zidani [1] reformulate a stochastic control problem with a state constraint as a target problem by introducing a conditional expectation process as a new controlled variable.…”

“…For λ < 1 there exist no optimal stopping times and for λ ≥ 1 stopping immediately τ = 0 is optimal. But for λ = 1 all stopping times τ a that embed the distributiona 2 δ −1 + (1 − a)δ 0 + a 2 δ 1 , a ∈ [0,1] into W are also optimal. It holds that E[τ a ] = a.…”

Duality and Approximation of Stochastic Optimal Control Problems under Expectation Constraints

“…The optimal control problem under state constraint has been studied by different papers (see e.g. [3] and the references therein), it can be formulated as follows:…”

“…), or one can use the so-called level-set approach to reformulate it into an optimization problem over a family of (unconstrained) singular control problems (see e.g. Bokanowski, Picarelli and Zidani [3] and the references therein). In the second main approach, one looks for necessary optimality conditions in the form of Pontryagin's maximum principle, involving a Lagrange multiplier, see e.g.…”

A Level-Set Approach for Stochastic Optimal Control Problems Under Controlled-Loss Constraints

“…This objective is achieved at the price of augmenting the state and control space by additional components and considering unbounded controls. More precisely, following the ideas developed in [24] for the case of a constraint holding pointwise in time almost surely, our approach relies on two main steps. The first one consists in building on the equivalence results developed in [25,26] to convert, by means of the martingale representation theorem, the original problem into a stochastic target problem involving almost-sure constraints and unbounded controls.…”