Necessary and Sufficient Optimality Conditions for Relaxed and Strict Control Problems

Abstract

We consider a stochastic control problem where the set of strict (classical) controls is not necessarily convex, and the system is governed by a nonlinear backward stochastic differential equation. By introducing a new approach, we establish necessary as well as sufficient conditions of optimality for two models. The first concerns the relaxed controls, who are measure-valued processes. The second is a particular case of the first and relates to strict control problems.The criteria to be minimized, over the set U, has the formwhere g and h are given functions and (y v t , z v t ) is the trajectory of the system controlled by v.A control u ∈ U is called optimal if it satisfiesStochastic control problems for backward and forward-backward systems have been studied by many authors including Peng [27], Xu [31], El-Karoui et al [12, 13], Wu [30], Dokuchaev and Zhou [9], Peng and Wu [28], Bahlali and Labed [2], Bahlali [5, 6], Shi and Wu [29], Ji and Zhou [19]. The dynamic programming approach was studied by Fuhrman and Tessitore [16] .Since the strict control domain being nonconvex, then if we use the classical method of spike variation on strict controls, the major difficulty in doing this is that the generator b and the running cost coefficient h depend on two variables y t and z t . Then, we can't derive directly the variational inequality, because z t is hard to handle, there is no convenient pointwise (in t) estimation for it, as opposed to the first variable y t . To overcome this difficulty, we introduce a new approach which consist to use a bigger new class R of processes by replacing the U -valued process (v t ) by a P (U )-valued process (q t ), where P (U ) is the space of probability measures on U equipped with the topology of stable convergence. This new class of processes is called relaxed controls and have a richer structure of compacity and convexity. This property of convexity of relaxed controls, enables us to treat the problem with the way of convex perturbation on relaxed controls.In the relaxed model, the system is governed by the BSDEThe functional cost to be minimized, over the class R of relaxed controls, is defined byA relaxed control µ is called optimal if it solvesThe relaxed control problem is a generalization of the problem of strict controls. Indeed, if q t (da) = δ vt (da) is a Dirac measure concentrated at aProof. Let u ∈ U and µ ∈ R be respectively a strict and relaxed controls such thatBy (39), we have J (µ) ≤ J (q) , ∀q ∈ R.Since δ (U) ⊂ R, thenHence,The control u becomes an element of U, then we getOn the other hand, by (40) we havedtµ t (da) stably, P − a.s.By (42), we get thenBy using (35) and letting n go to infinity in the above inequality, we getFinally, by (41) and (43), we haveThe lemma is proved.To establish necessary optimality conditions for strict controls, we need the following lemma Lemma 16 The strict control u minimizes J over U if and only if the relaxed control µ = δ u minimizes J over R. Proof. Suppose that u minimizes the cost J over U, then J (u) = inf v∈U J (v) ....

Optimality conditions of controlled backward doubly stochastic differential equations

“…The general case, where the control domain is not convex and the diffusion coefficient depends explicitly on the variable control, was solved by Peng [25] by introducing two adjoint processes and a variational inequality of second order. In a recent work, Bahlali [2] improves and generalizes all the previous results on the subject by introducing a new approach and derive necessary and sufficient optimality conditions in the general case, by using only the first order expansion and the associated adjoint equation.…”

Optimality conditions for stochastic boundary control problems governed by semilinear parabolic equations

“…In finite dimensional spaces, see [3][4][5][6][7][8][9][10][11][12][13][14], etc. For the cases of infinite dimensional spaces, to our knowledge, in general cases, there are very few papers to treat this problem.…”

The maximum principle in optimal control of systems driven by martingale measures

“…Let us point out that a different relaxation has been used in Bahlali (2008); Ahmed and Charalambous (2013), where the drift and diffusion coefficient have been replaced by their relaxed counterparts. Their relaxed state process is linear in the control variable and is different from ours, in the sense that in our case we relax the infinitesimal generator instead of relaxing directly the state process.…”

“…Let A = {a 1 , a 2 , ..., a n }, then every relaxed control dtq t (da) will be a convex combination of the Dirac measures dtq t (da) = n i=1 α i (t)dtδ ai (da) , where for each i, α i (t) is a real-valued process such that 0 ≤ α i (t) ≤ 1 and n i=1 α i (t) = 1. It is shown that the solution of the (relaxed) martingale problem (5) is the law of the solution of the following SDE (see Bahlali, 2008) …”

“…Moreover, thanks to Lemma 3.4 in Bahlali (2008) and Proposition 1, there exist a sequence (u n (t)) of strict controls and a Brownian motion B defined on an extension of (Ω, F, P ) such that for each t ∈ [0, T ] and each continuous bounded function ϕ from A to R,…”