Online adaptive optimal control for continuous-time nonlinear systems with completely unknown dynamics

Abstract

An online adaptive optimal control is proposed for continuous-time nonlinear systems with co mpletely unknown dynamics, which is achieved by developing a novel identifier-critic based approximate dynamic programming (ADP) algorith m with a dual neural network (NN) appro ximation structure. Firstly, an adaptive NN identifier is designed to obviate the requirement of complete knowledge of system dynamics, and a critic NN is emp loyed to approximate the optimal value function. Then the optimal control law is co mputed based on the informat ion fro m the identifier NN and the c rit ic NN so that the actor NN is not needed. In particu lar, a novel adaptive law design method with the parameter estimation error is proposed to online update the weights of both identifier NN and critic NN simu ltaneously, which converge to small neighborhoods of their ideal values. The closed-loop system stability and the convergence to small vicinity around the optimal solution are all proved by means of the Lyapunov theory. The proposed adaptation algorithm is also improved to achieve finite-time (FT) convergence of the NN weights. Finally, simulation results are provided to exemplify the efficacy of the proposed methods.

Adaptive optimal tracking control of unknown nonlinear systems using system augmentation

“…For specific continuous-time linear systems, the OTC may be designed by using Riccita equation method [1,2]. However, only a few results have been suggested for nonlinear systems because it is not trivial to solve the associated Hamilton-Jacobi-Bellman (HJB) equation [3]. Nevertheless, the direct application of dynamic programming (DP) [5] to solve OTC problem also encountered difficulties for high order systems.…”

“…In this paper, we study the optimal tracking control of nonlinear CT systems with completely unknown dynamics by further improving our previously proposed 'identifier-critic' strategy [3,4,9]. However, different to [4,9], the overall optimal control can be obtained simultaneously by using the the system augmentation method [12].…”

“…However, different to [4,9], the overall optimal control can be obtained simultaneously by using the the system augmentation method [12]. First, an adaptive identifier as [3,4,9] is used to estimate the unknown system dynamics. Then, an augmented system composed of the tracking error dynamics and the desired trajectory is constructed, and a new cost function for the augmented system is suggested.…”

Online Adaptive Optimal Control of Vehicle Active Suspension Systems Using Single‐Network Approximate Dynamic Programming

“…It has been proved in [16][17][18] that adaptive estimation method considering the parameter information can greatly improve the convergence speed in contrast to the conventional estimation method driven by the observer error. Inspired from these facts, a novel robust estimation method of is presented in the following analysis.…”

A Brief Review of Neural Networks Based Learning and Control and Their Applications for Robots

“…In [69], an ADP technique for online control and learning of a generalized multiple-input-multiple-output (MIMO) system was investigated. In [70], an adaptive NN based ADP control scheme was presented for a class of nonlinear systems with unknown dynamics. The optimal control law was calculated by using a dual neural network scheme with a critic NN and an identifier NN.…”