An Ergodic Control Problem for Constrained Diffusion Processes: Existence of Optimal Markov Control

Abstract

An ergodic control problem for a class of constrained diffusion processes is considered. The goal is the almost sure minimization of long term cost per unit time. The main result of the paper is that there exists an optimal Markov control for the considered problem. It is shown that under the assumption of regularity of the Skorohod map and appropriate conditions on the drift coefficient the class of controlled diffusion processes considered have strong, uniform in control, stability properties. The stability results on the class of controlled constrained diffusion processes considered in this work are non-trivial in that the domains are unbounded and the corresponding unconstrained diffusions are typically transient. These stability properties are key in obtaining appropriate tightness estimates. Once these estimates are available the remaining work lies in identifying weak limits of a certain family of occupation measures. In this regard an extension to the Echeverria-Weiss-Kurtz characterization of invariant measures of Markov processes, to the case of constrained-controlled processes considered in this paper, is proved. This characterization result is also crucially used in proving the compactness of the family of invariant measures of Markov processes corresponding to all possible Markov controls.

Ergodic Control for Constrained Diffusions: Characterization Using HJB Equations

“…Under these conditions, it follows via the Lipschitz property of the Skorohod map and the usual fixed point arguments that, (1.1) admits a unique strong solution (cf. Theorem 2.6 [8]). If X(·) solves (1.1) then (cf.…”

“…Under the standing assumptions of this paper there is a unique weak solution for (2.3) and denoting the law of the solution process X(·), when X(0) = x a.s., by P v x it can be shown that {P v x } x∈G is a strongly Feller Markov family (cf. Theorem 2.9 [8]). …”

“…In a recent work [8] an ergodic control problem for a class of constrained diffusion processes, in polyhedral cones, was studied. Such constrained diffusion processes are common in the heavy traffic analysis of queuing networks coming from problems in computer, communications and manufacturing systems.…”

“…Under such a control the solution to (1.1) becomes a Markov process and for this reason the map v(·) is referred to as a "Markov control". The main result of [8] is that, under appropriate conditions on the model, (Conditions 2.2, 2.4, 2.5, 2.8) there is a Markov control for which the infimum of the cost in (1.2) is attained.…”

“…

AbstractRecently in [8] an ergodic control problem for a class of diffusion processes, constrained to take values in a polyhedral cone, was considered. The main result of that paper was that under appropriate conditions on the model, there is a Markov control for which the infimum of the cost function is attained.

…”