No abstract

No abstract

We present a method for solving the Hamilton-Jacobi-Bellman (HJB) equation for a stochastic system with state constraints. A variable transformation is introduced which turns the HJB equation into a combination of a linear eigenvalue problem, a set of partial differential equations (PDE:s), and a point-wise equation. For a fixed solution to the eigenvalue problem, the PDE:s are linear and the point-wise equation is quadratic, indicating that the problem can be solved efficiently using an iterative scheme.As an example, we numerically solve for the optimal control of a Linear Quadratic Gaussian (LQG) system with state constraints. A reasonably accurate solution is obtained even with a very small number of collocation points (three in each dimension), which suggests that the method could be used on high order systems, mitigating the curse of dimensionality.

A method for finding optimal control policies for first order state-constrained, stochastic dynamic systems in continuous time is presented. The method relies on solution of the Hamilton-Jacobi-Bellman equation, which includes a diffusion term related to the stochastic disturbance in the model. A variable transformation is applied that turns the infinite-horizon optimal control problem into a linear eigenvalue problem in state-space. The method is demonstrated on a buffer control problem for a fuel cell-supercapacitor system. The obtained closed-form solution explains the shape of previous heuristically found control laws for this type of problem. Copyright