Risk-Sensitive Control on an Infinite Time Horizon

Fleming, Wendell H.; McEneaney, William M.

doi:10.1137/s0363012993258720

Cited by 300 publications

(176 citation statements)

References 31 publications

Supporting

Mentioning

174

Contrasting

Order By: Relevance

“…We briefly discuss the ergodic control problem of Section 3.3 that is known to be related to an elliptic eigenvalue problem [30,9,19]. In principle, the equivalence of (53) and (51) directly follows from the logarithmic transformation.…”

Section: B Ergodic Control Problemmentioning

confidence: 99%

Optimal control of multiscale systems using reduced-order models

Hartmann¹,

Latorre²,

Zhang³

et al. 2014

Journal of Computational Dynamics

View full text Add to dashboard Cite

We study optimal control of diffusions with slow and fast variables and address a question raised by practitioners: is it possible to first eliminate the fast variables before solving the optimal control problem and then use the optimal control computed from the reduced-order model to control the original, high-dimensional system? The strategy "first reduce, then optimize"-rather than "first optimize, then reduce"-is motivated by the fact that solving optimal control problems for high-dimensional multiscale systems is numerically challenging and often computationally prohibitive. We state sufficient and necessary conditions, under which the "first reduce, then control" strategy can be employed and discuss when it should be avoided. We further give numerical examples that illustrate the "first reduce, then optmize" approach and discuss possible pitfalls.

show abstract

Section: B Ergodic Control Problemmentioning

confidence: 99%

Optimal control of multiscale systems using reduced-order models

Hartmann¹,

Latorre²,

Zhang³

et al. 2014

Journal of Computational Dynamics

View full text Add to dashboard Cite

show abstract

“…The HJB equation (10) admits an interpretation in terms of an eigenvalue problem [37,12]. To reveal it we first of all note that…”

Section: Formulation As An Eigenvalue Problemmentioning

confidence: 99%

“…The type of optimal control problems that we consider here, and which appear relevant in molecular dynamics applications, belong to the class of ergodic stochastic control problems. Following ideas of Fleming and co-workers [12,13], we show that the optimal control of MD on an infinite time horizon can be reformulated as a linear eigenvalue problem that has deep relations to a Donsker-Varadhan large deviation principle (LDP) for the uncontrolled MD. The general strategy that is pursued in this article, namely, using low dimensional MSM for solving high dimensional optimal control problems is illustrated in Figure 1.…”

Section: Introductionmentioning

confidence: 98%

Optimal control of molecular dynamics using Markov state models

2012

View full text Add to dashboard Cite

A numerical scheme for solving high-dimensional stochastic control problems on an infinite time horizon that appear relevant in the context of molecular dynamics is outlined. The scheme rests on the interpretation of the corresponding Hamilton-Jacobi-Bellman equation as a nonlinear eigenvalue problem that, using a logarithmic transformation, can be recast as a linear eigenvalue problem, for which the principal eigenvalue and its eigenfunction are sought. The latter can be computed e ciently by approximating the underlying stochastic process with a coarse-grained Markov state model for the dominant metastable sets. We illustrate our method with two numerical examples, one of which involves the task of maximizing the population of ↵-helices in an ensemble of small biomolecules (Alanine dipeptide), and discuss the relation to the large deviation principle of Donsker and Varadhan.

show abstract

“…Condition (1.9) is precisely the result we will obtain here for the case of periodic diffusion (or reflected diffusions on a bounded region of R d ). The risk-sensitive control problem for diffusion processes (in various cases) has been studied by several authors, particularly in connection with robust control and differential games, for instance, we refer to Jacobson [7], Bensoussan and Van Schuppen [4], Whittle [12], Fleming and McEneaney [6], McEneaney [8], Nagai [9,10], Runolfsson [11].…”

Section: J(t X V)mentioning

confidence: 99%

Remarks on Risk-Sensitive Control Problems

Menaldi

Robin

2005

Appl Math Optim

View full text Add to dashboard Cite

The main purpose of this paper is to investigate the asymptotic behavior of the discounted risk-sensitive control problem for periodic diffusion processes when the discount factor α goes to zero. If u α (θ, x) denotes the optimal cost function, θ being the risk factor, then it is shown that lim α→0 αu α (θ, x) = ξ(θ) where ξ(θ) is the average on ]0, θ[ of the optimal cost of the (usual) infinite horizon risk-sensitive control problem.

show abstract

Risk-Sensitive Control on an Infinite Time Horizon

Cited by 300 publications

References 31 publications

Optimal control of multiscale systems using reduced-order models

Optimal control of multiscale systems using reduced-order models

Optimal control of molecular dynamics using Markov state models

Remarks on Risk-Sensitive Control Problems

Contact Info

Product

Resources

About