Occupation measures for controlled Markov processes: characterization and optimality

Bhatt, Abhay G.; Borkar, Vivek S.

doi:10.1214/aop/1065725192

Cited by 113 publications

(54 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We seek to obtain a full characterization of optimal controls via the study of the dual problem, and this leads to the HJB equation. For more details on this linear programming formulation see Section 4 in [9].…”

Section: And Letmentioning

confidence: 99%

Ergodic Control of a Class of Jump Diffusions with Finite Lévy Measures and Rough Kernels

Arapostathis

Caffarelli

Pang

et al. 2019

SIAM J. Control Optim.

View full text Add to dashboard Cite

We study the ergodic control problem for a class of jump diffusions in R d , which are controlled through the drift with bounded controls. The Lévy measure is finite, but has no particular structure-it can be anisotropic and singular. Moreover, there is no blanket ergodicity assumption for the controlled process. Unstable behavior is 'discouraged' by the running cost which satisfies a mild coercive hypothesis (i.e., is near-monotone). We first study the problem in its weak formulation as an optimization problem on the space of infinitesimal ergodic occupation measures, and derive the Hamilton-Jacobi-Bellman equation under minimal assumptions on the parameters, including verification of optimality results, using only analytical arguments. We also examine the regularity of invariant measures. Then, we address the jump diffusion model, and obtain a complete characterization of optimality.

show abstract

Section: And Letmentioning

confidence: 99%

Ergodic Control of a Class of Jump Diffusions with Finite Lévy Measures and Rough Kernels

Arapostathis

Caffarelli

Pang

et al. 2019

SIAM J. Control Optim.

View full text Add to dashboard Cite

show abstract

“…Arguably, this may require less computational efforts than the classic approach although, of course, more research is needed to validate this claim. 6. Generalization of Theorem 4.1 and a heuristic numerical algorithm.…”

Section: Vladimir Gaitsgory Alex Parkinson and Ilya Shvartsmanmentioning

confidence: 99%

“…The linear programming (LP) approach to optimal control problems has been studied extensively in both stochastic and deterministic settings (see, e.g., [6], [9], [10], [14], [25], [29], [33], [34] and, respectively, [1], [13], [15], [18], [19], [20], [22], [27], [28], [30], [31], [32], [36] as well as references therein). In particular, results establishing the validity of LP formulations of deterministic infinite horizon OC problems with time discounting have been obtained in [18], [19] and [27] for systems evolving in continuous time and in [16] and [17] for systems evolving in discrete time.…”

mentioning

confidence: 99%

Linear programming based optimality conditions and approximate solution of a deterministic infinite horizon discounted optimal control problem in discrete time

Gaitsgory¹,

Parkinson

Shvartsman

2019

Discrete &Amp; Continuous Dynamical Systems - B

View full text Add to dashboard Cite

It has been recently established that a deterministic infinite horizon discounted optimal control problem in discrete time is closely related to a certain infinite dimensional linear programming problem and its dual, the latter taking the form of a certain max-min problem. In the present paper, we use these results to establish necessary and sufficient optimality conditions for this optimal control problem and to investigate a way how the latter can be used for the construction of a near optimal control. 2010 Mathematics Subject Classification. Primary: 49N15, 49M29, 93C55.

show abstract

“…To overcome this limitation, we take a different approach to the ergodic control problem in (1.5). As well known, ergodic control problems can be cast as infinite dimensional linear programs [20,21]. Consider a controlled diffusion, with the control taking values in a space U with extended generator A, where the 'action' u ∈ U enters implicitly as a parameter in A.…”

Section: Introductionmentioning

confidence: 99%

“…Here P(R d × U) denotes the class of probability measures on the Borel σ-field of R d × U. Its elements are called ergodic occupation measures (see [20]). The dual problem takes the form…”

Section: Introductionmentioning

confidence: 99%

Strict monotonicity of principal eigenvalues of elliptic operators in Rd and risk-sensitive control

Arapostathis

Biswas

Saha

2019

Journal de Mathématiques Pures et Appliquées

View full text Add to dashboard Cite

This paper studies the eigenvalue problem on R d for a class of second order, elliptic operators of the form L f = a ij ∂ xi ∂ xj + b i ∂ xi + f , associated with non-degenerate diffusions. We show that strict monotonicity of the principal eigenvalue of the operator with respect to the potential function f fully characterizes the ergodic properties of the associated ground state diffusion, and the unicity of the ground state, and we present a comprehensive study of the eigenvalue problem from this point of view. This allows us to extend or strengthen various results in the literature for a class of viscous Hamilton-Jacobi equations of ergodic type with smooth coefficients to equations with measurable drift and potential. In addition, we establish the strong duality for the equivalent infinite dimensional linear programming formulation of these ergodic control problems. We also apply these results to the study of the infinite horizon risk-sensitive control problem for diffusions, and establish existence of optimal Markov controls, verification of optimality results, and the continuity of the controlled principal eigenvalue with respect to stationary Markov controls.

show abstract

Occupation measures for controlled Markov processes: characterization and optimality

Cited by 113 publications

References 7 publications

Ergodic Control of a Class of Jump Diffusions with Finite Lévy Measures and Rough Kernels

Ergodic Control of a Class of Jump Diffusions with Finite Lévy Measures and Rough Kernels

Linear programming based optimality conditions and approximate solution of a deterministic infinite horizon discounted optimal control problem in discrete time

Strict monotonicity of principal eigenvalues of elliptic operators in Rd and risk-sensitive control

Contact Info

Product

Resources

About