A radial basis collocation method for Hamilton–Jacobi–Bellman equations

Huang, Chih‐Feng; Wang, Song; Chen, C. S.; Li, Z. C.

doi:10.1016/j.automatica.2006.07.013

Cited by 24 publications

(22 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A collocation method using radial basis functions was proposed in [4]. A similar approach was used in [3], [1] for solving finite horizon HJB control equations. The work of Simpkins and Todorov [12] introduced collocation methods for solving infinite horizon robot control problems and was inspirational to us, however, the approach was not directly applicable to the finite horizon problems we consider here.…”

Section: Relation To Prior Workmentioning

confidence: 99%

“…The objective function minimized in [12], [3], [1], [15], [14] is the squared difference between the the lefthand and righthand side of the HJB equation. For the terminal time T we find the parameters w T by minimizing the sum of squared differences between the terminal reward and the value function estimate at the set of collocation points, x T , at time T :…”

Section: Collocation For Computing An Approximately Optimal Contrmentioning

confidence: 99%

“…While previous approaches [2], [11] have sought to address the local minima problem in various ways, these approaches are not applicable to the general obstacle avoidance problem; focusing instead on special cases (see Section VI for a more thorough discussion of the existing literature). Here we show that with an appropriate function approximation scheme, collocation approaches designed to compute optimal controllers for nonlinear systems [12], [3], [1], [15], [14] can effectively solve the obstacle avoidance problem.…”

Section: Introductionmentioning

confidence: 99%

“…Recently, there has been a surge of interest in collocation methods for solving continuous state and action control problems [12], [3], [1], [15], [14]. These algorithms work by computing an approximate solution to the Hamilton Jacobi Bellman (HJB) Equation (a sufficient condition for deriving an optimal control policy) at a finite set of states.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Control by Gradient Collocation: Applications to optimal obstacle avoidance and minimum torque control

Ruvolo

Movellan

2012

2012 IEEE/RSJ International Conference on Intelligent Robots and Systems

View full text Add to dashboard Cite

Abstract-We present a new machine learning algorithm for learning optimal feedback control policies to guide a robot to a goal in the presence of obstacles. Our method works by first reducing the problem of obstacle avoidance to a continuous state, action, and time control problem, and then uses efficient collocation methods to solve for an optimal feedback control policy. This formulation of the obstacle avoidance problem improves over standard approaches, such as potential field methods, by being resistant to local minima, allowing for moving obstacles, handling stochastic systems, and computing feedback control strategies that take into account the robot's (possibly non-linear) dynamics. In addition to contributing a new method for obstacle avoidance, our work contributes to the state-of-the-art in collocation methods for non-linear stochastic optimal control problems in two important ways: (1) we show that taking into account local gradient and secondorder derivative information of the optimal value function at the collocation points allows us to exploit knowledge of the derivative information about the system dynamics, and (2) we show that computational savings can be achieved by directly fitting the gradient of the optimal value function rather than the optimal value function itself. We validate our approach on three problems: non-convex obstacle avoidance of a point-mass robot, obstacle avoidance for a 2 degree of freedom robotic manipulator, and optimal control of a non-linear dynamical system.

show abstract

Section: Relation To Prior Workmentioning

confidence: 99%

Section: Collocation For Computing An Approximately Optimal Contrmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Control by Gradient Collocation: Applications to optimal obstacle avoidance and minimum torque control

Ruvolo

Movellan

2012

2012 IEEE/RSJ International Conference on Intelligent Robots and Systems

View full text Add to dashboard Cite

show abstract

“…The HJB equation can be solved using the upwind finite volume method in Wang, Jennings, and Teo (2003). In Huang, Wang, Chen, and Li (2006), a collocation method was developed to solve the same HJB equation using Radial Basis Functions (RBFs). In Alwardi, Wang, Jennings, and Richardson (2012), this method was extended with an adaptive scheme that refines the distribution of the RBF centers by means of feeding back the approximation error.…”

Section: Introductionmentioning

confidence: 99%

A sparse collocation method for solving time-dependent HJB equations using multivariate B-splines

2014

View full text Add to dashboard Cite

a b s t r a c tThis paper presents a sparse collocation method for solving the time-dependent Hamilton-Jacobi-Bellman (HJB) equation associated with the continuous-time optimal control problem on a fixed, finite timehorizon with integral cost functional. Through casting the problem in a recursive framework using the value-iteration procedure, the value functions of every iteration step is approximated with a time-varying multivariate simplex B-spline on a certain state domain of interest. In the collocation scheme, the timedependent coefficients of the spline function are further approximated with ordinary univariate B-splines to yield a discretization for the value function fully in terms of piece-wise polynomials. The B-spline coefficients are determined by solving a sequence of highly sparse quadratic programming problems. The proposed algorithm is demonstrated on a pair of benchmark example problems. Simulation results indicate that the method can yield increasingly more accurate approximations of the value function by refinement of the triangulation.

show abstract

A Computational Method for Stochastic Optimal Control Problems in Financial Mathematics

Kafash

Delavarkhalafi

Karbassi

2015

Asian Journal of Control

View full text Add to dashboard Cite

Principle of optimality or dynamic programming leads to derivation of a partial differential equation (PDE) for solving optimal control problems, namely the Hamilton‐Jacobi‐Bellman (HJB) equation. In general, this equation cannot be solved analytically; thus many computing strategies have been developed for optimal control problems. Many problems in financial mathematics involve the solution of stochastic optimal control (SOC) problems. In this work, the variational iteration method (VIM) is applied for solving SOC problems. In fact, solutions for the value function and the corresponding optimal strategies are obtained numerically. We solve a stochastic linear regulator problem to investigate the applicability and simplicity of the presented method and prove its convergence. In particular, for Merton's portfolio selection model as a problem of portfolio optimization, the proposed numerical method is applied for the first time and its usefulness is demonstrated. For the nonlinear case, we investigate its convergence using Banach's fixed point theorem. The numerical results confirm the simplicity and efficiency of our method.

show abstract

A radial basis collocation method for Hamilton–Jacobi–Bellman equations

Cited by 24 publications

References 4 publications

Control by Gradient Collocation: Applications to optimal obstacle avoidance and minimum torque control

Control by Gradient Collocation: Applications to optimal obstacle avoidance and minimum torque control

A sparse collocation method for solving time-dependent HJB equations using multivariate B-splines

A Computational Method for Stochastic Optimal Control Problems in Financial Mathematics

Contact Info

Product

Resources

About