Ergodic Control for Constrained Diffusions: Characterization Using HJB Equations

Abstract

Recently in [8] an ergodic control problem for a class of diffusion processes, constrained to take values in a polyhedral cone, was considered. The main result of that paper was that under appropriate conditions on the model, there is a Markov control for which the infimum of the cost function is attained. In the current work we characterize the value of the ergodic control problem via a suitable Hamilton-Jacobi-Bellman (HJB) equation. The theory of existence and uniqueness of classical solutions, for PDEs in domains with corners and reflection fields which are oblique, discontinuous and multi-valued on corners, is not available. We show that the natural HJB equation for the ergodic control problem admits a unique continuous viscosity solution which enables us to characterize the value function of the control problem. The existence of a solution to this HJB equation is established via the classical vanishing discount argument. The key step is proving the pre-compactness of the family of suitably re-normalized discounted value functions. In this regard we use a recent technique, introduced in [4], of using the Athreya-Ney-Nummelin pseudo-atom construction for obtaining a coupling of a pair of embedded, discrete time, controlled Markov chains.

The Eigenvalue Problem of Singular Ergodic Control

“…This approach essentially amounts to replacing in (1.1) with ıu and studying the resulting PDE for ı > 0 and small. If this resulting PDE has a unique solution u ı , the hope is that there is a sequence of ı tending to 0 such that ıu ı tends to : However, we believe the earliest application of this method appears in periodic homogenization of viscosity solutions of PDE in [7,8,15]; for this approach applied to general ergodic control problems, see reference [4].…”

Nonexistence of nonconstant solutions of some degenerate Bellman equations and applications to stochastic control

“…Moreover χ ∈ C 2 (Ω) and is unique up to addition of constants among all solutions to (3) which satisfy (16).…”

“…In this context, our condition on the behavior of the operator near the boundary ensures some invariance property of the domain for the associated controlled diffusion process.T 0 l(X α· t , α t )dt as T → +∞, and Dχ allows to synthesise an optimal feedback (at least in principle, under further assumptions). There is a large literature on this topic, see [28,14,16,11] for diffusions reflected at the boundary and Neumann boundary conditions in (3), [3, 1] for periodic boundary conditions, and the recent monograph [2], as well as the references therein. Some of the cited papers deal with the model problemin Ω, for p > 1, which is a special case of (3) with unbounded drift b and running cost l, see, e.g., [28].…”

On Viscosity Solution of HJB Equations with State Constraints and Reflection Control

“…The authors of [2,7,8] study viscosity solution in a non-smooth domain but with linear Neumann type boundary condition. In [5] viscosity solution framework is used to characterize the value function of an ergodic control problem with dynamics given by controlled constrained diffusions in a polyhedral domain. Nonlinear Neumann boundary condition is studied in [3,9] (see also the references therein) for domains with regular boundary (at least C 1 ).…”