On the infinite time solution to state-constrained stochastic optimal control problems

Rutquist, Per; Breitholtz, Claes; Wik, Torsten

doi:10.1016/j.automatica.2007.10.018

Cited by 17 publications

(17 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, we require both W and R to be positive definite and bounded everywhere on Ω, but otherwise impose no restrictions on them. Contrary to the assumptions in previous work [9,10,14] and the work of Kappen [7] and Broek et al [12] they are no longer required to relate to the inverse of eachother. As formulated, the control u and the noise w enters the state equation via the same matrix G. However, the problem can easily be reformulated such that the control and noise enter via different matrices as long as they have the same column space [14].…”

Section: Problem Formulationcontrasting

confidence: 40%

“…The nonlinear optimization solver minimizes φ at the same time as it solves (16), (14) and (10). As a starting guess for K we can use (17), and as a starting guess for Z we can solve (14) for ρ = 0, which is a linear eigenvalue problem given our starting guess for K.…”

Section: Solving the Problemmentioning

confidence: 99%

“…affecting both states here. (The case with only one input is easier, and can be solved using an exact linearization of the HJB equation as in [10]. )…”

Section: Numeric Examplementioning

confidence: 99%

“…However, the resulting partial differential equation (PDE) is nonlinear and difficult to solve as it will in general also have infinite boundary conditions. For systems where both the input and the (white noise) disturbances enters affinely, a logarithmic transformation [4] can be used to achieve an exact linearization of the HJB equation [8,9,10,14].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Solving the Hamilton-Jacobi-Bellman equation for a stochastic system with state constraints

Rutquist

Wik

Breitholtz

2014

53rd IEEE Conference on Decision and Control

Self Cite

View full text Add to dashboard Cite

We present a method for solving the Hamilton-Jacobi-Bellman (HJB) equation for a stochastic system with state constraints. A variable transformation is introduced which turns the HJB equation into a combination of a linear eigenvalue problem, a set of partial differential equations (PDE:s), and a point-wise equation. For a fixed solution to the eigenvalue problem, the PDE:s are linear and the point-wise equation is quadratic, indicating that the problem can be solved efficiently using an iterative scheme.As an example, we numerically solve for the optimal control of a Linear Quadratic Gaussian (LQG) system with state constraints. A reasonably accurate solution is obtained even with a very small number of collocation points (three in each dimension), which suggests that the method could be used on high order systems, mitigating the curse of dimensionality.

show abstract

Section: Problem Formulationcontrasting

confidence: 40%

Section: Solving the Problemmentioning

confidence: 99%

“…affecting both states here. (The case with only one input is easier, and can be solved using an exact linearization of the HJB equation as in [10]. )…”

Section: Numeric Examplementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Solving the Hamilton-Jacobi-Bellman equation for a stochastic system with state constraints

Rutquist

Wik

Breitholtz

2014

53rd IEEE Conference on Decision and Control

Self Cite

View full text Add to dashboard Cite

show abstract

“…Dealing with stochastic systems, such investigation has been usually carried out in the framework of a complete knowledge of the state of the system (i.e., the state is directly available with no need for outputs providing possibly noisy/incomplete measurements of the state). Usual difficulties involve the solution of the Hamilton-Jacobi-Bellman (HJB) equations associated to the optimal control problem: in [5], the stochastic HJB equation is iteratively solved with successive approximations; in [6], the infinitetime HJB equation is reformulated as an eigenvalue problem; in [7], a transformation approach is proposed for solving the HJB equation arising in quadratic-cost control for nonlinear deterministic and stochastic systems. Finally, in a pair of recent papers, a solution to the nonlinear HJB equation is provided, by expressing it in the form of decoupled Forward and Backward Stochastic Differential Equations (FBSDEs), for an L 2 -and an L 1 -type optimal control setting (see [8,9], respectively).…”

Section: Introductionmentioning

confidence: 99%

A Stochastic Optimal Regulator for a Class of Nonlinear Systems

Mavelli

Palombo

Palumbo

2019

Mathematical Problems in Engineering

View full text Add to dashboard Cite

This work investigates an optimal control problem for a class of stochastic differential bilinear systems, affected by a persistent disturbance provided by a nonlinear stochastic exogenous system (nonlinear drift and multiplicative state noise). The optimal control problem aims at minimizing the average value of a standard quadratic-cost functional on a finite horizon. It has been supposed that neither the state of the system nor the state of the exosystem is directly measurable (incomplete information case). The approach is based on the Carleman embedding, which allows to approximate the nonlinear stochastic exosystem in the form of a bilinear system (linear drift and multiplicative noise) with respect to an extended state that includes the state Kronecker powers up to a chosen degree. This way the stochastic optimal control problem may be restated in a bilinear setting and the optimal solution is provided among all the affine transformations of the measurements. The present work is a nontrivial extension of previous work of the authors, where the Carleman approach was exploited in a framework where only additive noises had been conceived for the state and for the exosystem. Numerical simulations support theoretical results by showing the improvements in the regulator performances by increasing the order of the approximation.

show abstract

Optimal control applications and methods literature survey (No. 8)

2008

Optim Control Appl Methods

View full text Add to dashboard Cite

On the infinite time solution to state-constrained stochastic optimal control problems

Cited by 17 publications

References 6 publications

Solving the Hamilton-Jacobi-Bellman equation for a stochastic system with state constraints

Solving the Hamilton-Jacobi-Bellman equation for a stochastic system with state constraints

A Stochastic Optimal Regulator for a Class of Nonlinear Systems

Optimal control applications and methods literature survey (No. 8)

Contact Info

Product

Resources

About