Approximate Optimal Control of Affine Nonlinear Continuous-Time Systems Using Event-Sampled Neurodynamic Programming

Sahoo, Avimanyu; Xu, Hao; Jagannathan, S.

doi:10.1109/tnnls.2016.2539366

Cited by 60 publications

(25 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that (x k ) is actually a discretized value of (x). Moreover, as illustrated in other works, [22][23][24][25][26][27] the time-triggered HJB equation lays a foundation for developing the event-triggered HJB equation. In what follows, we recall the time-triggered HJB equation for the auxiliary system.…”

Section: Event-triggered Robust Control Strategymentioning

confidence: 91%

“…Remark 3. The triggering instant t k can be obtained via the triggering condition (24). Thus, the minimal intersample time (Δt k ) min is available (note: Δt k = t k+1 − t k , ie, k ∈ N).…”

Section: Event-triggered Robust Controller Design Via Solving the Evementioning

confidence: 99%

“…Remark 5. If letting = , then the triggering thresholdē T given in (45) is the same as the triggering threshold e T given in (24). For simplicity, in this paper, we choose = .…”

Section: Stability Analysismentioning

confidence: 99%

See 2 more Smart Citations

An adaptive critic approach to event‐triggered robust control of nonlinear systems with unmatched uncertainties

Yang

Wei

2018

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

In this paper, we develop a novel event-triggered robust control strategy for continuous-time nonlinear systems with unmatched uncertainties. First, we build a relationship to show that the event-triggered robust control can be obtained by solving an event-triggered nonlinear optimal control problem of the auxiliary system. Then, within the framework of reinforcement learning, we propose an adaptive critic approach to solve the event-triggered nonlinear optimal control problem. Unlike typical actor-critic dual approximators used in reinforcement learning, we employ a unique critic approximator to derive the solution of the event-triggered Hamilton-Jacobi-Bellman equation arising in the nonlinear optimal control problem. The critic approximator is updated via the gradient descent method, and the persistence of excitation condition is necessary. Meanwhile, under a newly proposed event-triggering condition, we prove that the developed critic approximator update rule guarantees all signals in the auxiliary closed-loop system to be uniformly ultimately bounded. Moreover, we demonstrate that the obtained event-triggered optimal control can ensure the original system to be stable in the sense of uniform ultimate boundedness. Finally, a F-16 aircraft plant and a nonlinear system are provided to validate the present event-triggered robust control scheme.

show abstract

Section: Event-triggered Robust Control Strategymentioning

confidence: 91%

Section: Event-triggered Robust Controller Design Via Solving the Evementioning

confidence: 99%

See 1 more Smart Citation

An adaptive critic approach to event‐triggered robust control of nonlinear systems with unmatched uncertainties

Yang

Wei

2018

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

show abstract

“…Due to this characteristic of ADP, the phenomenon of 'the curse of dimensionality' is overcome [12]. There are many synonyms proposed for ADP, such as 'ADP' [13][14][15][16][17][18][19], 'approximate dynamic programming' [20][21][22], 'adaptive critic designs' [23], 'neurodynamic programming' [24,25], and 'reinforcement learning (RL)' [26,27].…”

Section: Introductionmentioning

confidence: 99%

Adaptive dynamic programming for robust neural control of unknown continuous‐time non‐linear systems

Yang

Liu

et al. 2017

IET Control Theory & Applications

View full text Add to dashboard Cite

The design of robust controllers for continuous-time (CT) non-linear systems with completely unknown non-linearities is a challenging task. The inability to accurately identify the non-linearities online or offline motivates the design of robust controllers using adaptive dynamic programming (ADP). In this study, an ADP-based robust neural control scheme is developed for a class of unknown CT non-linear systems. To begin with, the robust non-linear control problem is converted into a non-linear optimal control problem via constructing a value function for the nominal system. Then an ADP algorithm is developed to solve the non-linear optimal control problem. The ADP algorithm employs actor-critic dual networks to approximate the control policy and the value function, respectively. Based on this architecture, only system data is necessary to update simultaneously the actor neural network (NN) weights and the critic NN weights. Meanwhile, the persistence of excitation assumption is no longer required by using the Monte Carlo integration method. The closed-loop system with unknown non-linearities is demonstrated to be asymptotically stable under the obtained optimal control. Finally, two examples are provided to validate the developed method.

show abstract

“…During the last decades, many researchers have made much efforts to overcome the problem of "curse of dimensionality". Based on dynamic programming method, a large number of reinforcement learning (RL) methods such as approximate dynamic programming (ADP), Neuro-dynamic programming (NDP) and adaptive dynamic programming (ADP) have thus been proposed to deal well with the problem of "curse of dimensionality" problem [36,11,8,30,41,45,40,23,38,2,3]. In the process of various RL methods emerged, optimal control also realizes the transition from model-based reinforcement learning to model-free reinforcement learning.…”

mentioning

confidence: 99%

Finite-horizon optimal control of discrete-time linear systems with completely unknown dynamics using Q-learning

Zhao¹,

Zhang²

2021

Journal of Industrial &Amp; Management Optimization

View full text Add to dashboard Cite

This paper investigates finite-horizon optimal control problem of completely unknown discrete-time linear systems. The completely unknown here refers to that the system dynamics are unknown. Compared with infinitehorizon optimal control, the Riccati equation (RE) of finite-horizon optimal control is time-dependent and must meet certain terminal boundary constraints, which brings the greater challenges. Meanwhile, the completely unknown system dynamics have also caused additional challenges. The main innovation of this paper is the developed cyclic fixed-finite-horizon-based Q-learning algorithm to approximate the optimal control input without requiring the system dynamics. The developed algorithm main consists of two phases: the data collection phase over a fixed-finite-horizon and the parameters update phase. A least-squares method is used to correlate the two phases to obtain the optimal parameters by cyclic. Finally, simulation results are given to verify the effectiveness of the proposed cyclic fixed-finite-horizon-based Q-learning algorithm.2010 Mathematics Subject Classification. Primary: 58F15, 58F17; Secondary: 53C35.

show abstract

Approximate Optimal Control of Affine Nonlinear Continuous-Time Systems Using Event-Sampled Neurodynamic Programming

Cited by 60 publications

References 21 publications

An adaptive critic approach to event‐triggered robust control of nonlinear systems with unmatched uncertainties

An adaptive critic approach to event‐triggered robust control of nonlinear systems with unmatched uncertainties

Adaptive dynamic programming for robust neural control of unknown continuous‐time non‐linear systems

Finite-horizon optimal control of discrete-time linear systems with completely unknown dynamics using Q-learning

Contact Info

Product

Resources

About