An adaptive least-squares collocation radial basis function method for the HJB equation

Abstract

We present a novel numerical method for the Hamilton-Jacobi-Bellman equation governing a class of optimal feedback control problems. The spatial discretization is based on a least-squares collocation Radial Basis Function method and the time discretization is the backward Euler finite difference. A stability analysis is performed for the discretization method. An adaptive algorithm is proposed so that at each time step, the approximate solution can be constructed recursively and optimally. Numerical results are presented to demonstrate the efficiency and accuracy of the method

Control by Gradient Collocation: Applications to optimal obstacle avoidance and minimum torque control

“…That is we assume that we are given a, b, and c in Equation 1. Additionally, we assume that the quadratic penalty on the control signal, q t , is given.…”

“…The objective function minimized in [12], [3], [1], [15], [14] is the squared difference between the the lefthand and righthand side of the HJB equation. For the terminal time T we find the parameters w T by minimizing the sum of squared differences between the terminal reward and the value function estimate at the set of collocation points, x T , at time T :…”

“…A collocation method using radial basis functions was proposed in [4]. A similar approach was used in [3], [1] for solving finite horizon HJB control equations. The work of Simpkins and Todorov [12] introduced collocation methods for solving infinite horizon robot control problems and was inspirational to us, however, the approach was not directly applicable to the finite horizon problems we consider here.…”

“…While previous approaches [2], [11] have sought to address the local minima problem in various ways, these approaches are not applicable to the general obstacle avoidance problem; focusing instead on special cases (see Section VI for a more thorough discussion of the existing literature). Here we show that with an appropriate function approximation scheme, collocation approaches designed to compute optimal controllers for nonlinear systems [12], [3], [1], [15], [14] can effectively solve the obstacle avoidance problem.…”

“…Recently, there has been a surge of interest in collocation methods for solving continuous state and action control problems [12], [3], [1], [15], [14]. These algorithms work by computing an approximate solution to the Hamilton Jacobi Bellman (HJB) Equation (a sufficient condition for deriving an optimal control policy) at a finite set of states.…”