In this paper, a value iteration adaptive dynamic programming (ADP) algorithm is developed to solve infinite horizon undiscounted optimal control problems for discrete-time nonlinear systems. The present value iteration ADP algorithm permits an arbitrary positive semi-definite function to initialize the algorithm. A novel convergence analysis is developed to guarantee that the iterative value function converges to the optimal performance index function. Initialized by different initial functions, it is proven that the iterative value function will be monotonically nonincreasing, monotonically nondecreasing, or nonmonotonic and will converge to the optimum. In this paper, for the first time, the admissibility properties of the iterative control laws are developed for value iteration algorithms. It is emphasized that new termination criteria are established to guarantee the effectiveness of the iterative control laws. Neural networks are used to approximate the iterative value function and compute the iterative control law, respectively, for facilitating the implementation of the iterative ADP algorithm. Finally, two simulation examples are given to illustrate the performance of the present method.

No abstract

No abstract

Summary In this paper, a novel identifier–actor–critic optimal control scheme is developed for discrete‐time affine nonlinear systems with uncertainties. In contrast to traditional adaptive dynamic programming methodology, which requires at least partial knowledge of the system dynamics, a neural‐network identifier is employed to learn the unknown control coefficient matrix working together with actor–critic‐based scheme to solve the optimal control online. The critic network learns the approximate value function at each step. The actor network attempts to improve the current policy based on the approximate value function. The weights of all neural networks are updated at each sampling instant. Lyapunov theory is utilized to prove the stability of closed‐loop system. It shows that system states and neural network weights are uniformly ultimately bounded. Finally, simulations are provided to illustrate the effectiveness of the developed method. Copyright © 2016 John Wiley & Sons, Ltd.

No abstract

No abstract