Stochastic optimal control of state constrained systems

Broek, Bart van den; Wiegerinck, Wim; Kappen, Bert

doi:10.1080/00207179.2011.565520

Cited by 14 publications

(11 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, we require both W and R to be positive definite and bounded everywhere on Ω, but otherwise impose no restrictions on them. Contrary to the assumptions in previous work [9,10,14] and the work of Kappen [7] and Broek et al [12] they are no longer required to relate to the inverse of eachother. As formulated, the control u and the noise w enters the state equation via the same matrix G. However, the problem can easily be reformulated such that the control and noise enter via different matrices as long as they have the same column space [14].…”

Section: Problem Formulationcontrasting

confidence: 43%

See 1 more Smart Citation

Solving the Hamilton-Jacobi-Bellman equation for a stochastic system with state constraints

Rutquist

Wik

Breitholtz

2014

53rd IEEE Conference on Decision and Control

View full text Add to dashboard Cite

We present a method for solving the Hamilton-Jacobi-Bellman (HJB) equation for a stochastic system with state constraints. A variable transformation is introduced which turns the HJB equation into a combination of a linear eigenvalue problem, a set of partial differential equations (PDE:s), and a point-wise equation. For a fixed solution to the eigenvalue problem, the PDE:s are linear and the point-wise equation is quadratic, indicating that the problem can be solved efficiently using an iterative scheme.As an example, we numerically solve for the optimal control of a Linear Quadratic Gaussian (LQG) system with state constraints. A reasonably accurate solution is obtained even with a very small number of collocation points (three in each dimension), which suggests that the method could be used on high order systems, mitigating the curse of dimensionality.

show abstract

Section: Problem Formulationcontrasting

confidence: 43%

“…It is worth noting that the solutions to (12) and (16) do not depend on the magnitude of ∇Z, only its direction. Also worth noting is that (16) is satisfied if the columns of G are linearly independent, and…”

Section: Variable Transformationmentioning

confidence: 99%

Solving the Hamilton-Jacobi-Bellman equation for a stochastic system with state constraints

Rutquist

Wik

Breitholtz

2014

53rd IEEE Conference on Decision and Control

View full text Add to dashboard Cite

show abstract

“…In other words, policies will be deemed more likely if they bring about states that conform to prior preferences. In the optimal control literature, this part of expected free energy underwrites KL control ( Todorov, 2008 , van den Broek et al, 2010 ). In economics, it leads to risk sensitive policies ( Fleming & Sheu, 2002 ).…”

Section: Properties Of the Expected Free Energymentioning

confidence: 99%

Active inference on discrete state-spaces: A synthesis

Costa

Parr

Sajid

et al. 2020

Journal of Mathematical Psychology

174

203

View full text Add to dashboard Cite

Active inference is a normative principle underwriting perception, action, planning, decision-making and learning in biological or artificial agents. From its inception, its associated process theory has grown to incorporate complex generative models, enabling simulation of a wide range of complex behaviours. Due to successive developments in active inference, it is often difficult to see how its underlying principle relates to process theories and practical implementation. In this paper, we try to bridge this gap by providing a complete mathematical synthesis of active inference on discrete state-space models. This technical summary provides an overview of the theory, derives neuronal dynamics from first principles and relates this dynamics to biological processes. Furthermore, this paper provides a fundamental building block needed to understand active inference for mixed generative models; allowing continuous sensations to inform discrete representations. This paper may be used as follows: to guide research towards outstanding challenges, a practical guide on how to implement active inference to simulate experimental behaviour, or a pointer towards various in-silico neurophysiological responses that may be used to make empirical predictions.

show abstract

“…In certain cases, the path integral is computable numerically using Laplace approximations or Monte Carlo sampling. Different applications of path-integral stochastic optimal control have been explored, such as reinforcement learning [36], variable stiffness control (equivalent to automatic tuning of PD gains) [37] and risk sensitive control [38]. The main issue with path integrals is that for most nonlinear systems the solution is computationally demanding and can not be obtained in real-time on existing processors.…”

Section: Introductionmentioning

confidence: 99%

Stochastic receding horizon control for robots with probabilistic state constraints

Shah

Pahlajani

Lacock³

et al. 2012

2012 IEEE International Conference on Robotics and Automation

View full text Add to dashboard Cite

This paper presents a receding horizon control design for a robot subject to stochastic uncertainty, moving in a constrained environment. Instead of minimizing the expectation of a cost functional while ensuring satisfaction of probabilistic state constraints, we propose a two-stage solution where the path that minimizes the cost functional is planned deterministically, and a local stochastic optimal controller with exit constraints ensures satisfaction of probabilistic state constraints while following the planned path. This control design strategy ensures boundedness of errors around the reference path and collision-free convergence to the goal with probability one under the assumption of unbounded inputs. We show that explicit expressions for the control law are possible for certain cases. We provide simulation results for a point robot moving in a constrained two-dimensional environment under Brownian noise. The method can be extended to systems with bounded inputs, if a small nonzero probability of failure can be accepted.

show abstract

Stochastic optimal control of state constrained systems

Cited by 14 publications

References 29 publications

Solving the Hamilton-Jacobi-Bellman equation for a stochastic system with state constraints

Solving the Hamilton-Jacobi-Bellman equation for a stochastic system with state constraints

Active inference on discrete state-spaces: A synthesis

Stochastic receding horizon control for robots with probabilistic state constraints

Contact Info

Product

Resources

About