Optimal Output Feedback Control of Nonlinear Partially-Unknown Constrained-Input Systems Using Integral Reinforcement Learning

Event‐triggered‐based integral reinforcement learning output feedback optimal control for partially unknown constrained‐input nonlinear systems

“…Proof The proof that the state estimation error $$$ \tilde{x} $$$ and the NN weight estimation error $$$ {\tilde{W}}_f $$$ are UUB does not much differ from that of [17, 19] and thus it is omitted here for brevity. □…”

“…In [4], an adaptive synchronous PI algorithm with actor-critic architecture is developed, which can adjust the critic and actor NNs simultaneously and can be called synchronous IRL (SIRL). In [17], SIRL algorithm based on neural network observer (NNO) is designed to solve HJB equation for the nonlinear system with the unknown drift dynamics and unmeasurable state. In practical application, if the input saturation of the actuator is not considered, the designed controller may lead to system instability or worse stability, so more and more researchers have studied the problem of the actuator saturation in the design of optimal control algorithms [12,18,19].…”

“…Compared with the dynamics identification method proposed in [11], we adopt the NNO to estimate unmeasurable system state online only by the input-output data of the system, which has good practical application value. Compared with the time-triggered control schemes mentioned above, such as [16][17][18][19], the event-triggered-based control algorithm proposed in this paper can reduce computation and communication burden and can reduce the update frequency of the controller. Compared with ADP-based event-triggered control approaches proposed in [27,28], the prior knowledge of drift dynamics in this paper is relaxed, and the system state is regarded as unmeasurable.…”

“…RL based on policy iteration (PI) technology is an effective method to deal with the optimization problems, and PI technology is implemented by alternating actions of policy evaluation and policy improvement with critic–actor architecture, where two kinds of neural networks (NNs) as the critic NN and the actor NN are used to approximate the optimal cost function and optimal control policy, respectively [14, 15]. As an improvement of RL algorithm, integral RL (IRL) has been investigated in [4, 16, 17]. In [16], a novel PI algorithm with critic‐actor architecture considered as IRL is proposed to solve the optimal control problem by introducing integral Bellman equation that the knowledge of system internal dynamic is no longer required, and the integral term in policy evaluation step can be addressed as the reinforcement signal over the time interval $$$ \left[t,t+T\right) $$$.…”

A neural network approach for improving airfoil active flutter suppression under control-input constraints

“…Although control-input constraints (CICs) have been considered in some AFS studies Ko et al, 2002;Viswamurthy and Ganguli, 2008;Wang et al, 2011), none of the existing solutions address the problem in the sense of optimal control. Despite numerous methods of nonlinear optimal control online synthesis (NOCOS) for systems with CICs being available (Liang et al, 2019;Na et al, 2019;Ren et al, 2019), these methods are inapplicable to AFS because of problems related to stability, application scope, and real-time implementation. Moreover, these methods are limited to locally parameter-invariant nonlinear systems, whereas aeroelastic systems are parameter varying as the dynamics also change nonlinearly with the freestream airflow speed.…”