Decentralized Stabilization for a Class of Continuous-Time Nonlinear Interconnected Systems Using Online Learning Optimal Control Approach

Abstract

Abstract-In this paper, using a neural-network-based online learning optimal control approach, a novel decentralized control strategy is developed to stabilize a class of continuoustime nonlinear interconnected large-scale systems. First, optimal controllers of the isolated subsystems are designed with cost functions reflecting the bounds of interconnections. Then, it is proven that the decentralized control strategy of the overall system can be established by adding appropriate feedback gains to the optimal control policies of the isolated subsystems. Next, an online policy iteration algorithm is presented to solve the Hamilton-Jacobi-Bellman equations related to the optimal control problem. Through constructing a set of critic neural networks, the cost functions can be obtained approximately, followed by the control policies. Furthermore, the dynamics of the estimation errors of the critic networks are verified to be uniformly and ultimately bounded. Finally, a simulation example is provided to illustrate the effectiveness of the present decentralized control scheme.Index Terms-Adaptive dynamic programming, decentralized control, large-scale systems, neural networks, nonlinear interconnected systems, optimal control, policy iteration, reinforcement learning.

Resilient Autonomous Control of Distributed Multiagent Systems in Contested Environments

“…The main bottleneck in deploying successful distributed MASs is designing secure control protocols that can learn about system uncertainties while showing some level of functionality in the presence of cyber-physical attacks. Reinforcement learning (RL) [7]- [9], inspired by learning mechanisms observed in mammals, has been successfully used to learn optimal solutions online in single agent systems for both regulation and tracking control problems [10]- [16] and recently for MASs [17]- [19]. Existing RL-based controllers for leader-follower MASs assume that the leader is passive and without any control input.…”

Observer‐based fuzzy adaptive optimal control for nonlinear continuous‐time saturation interconnected systems

“…Recently, the optimal control problems of nonlinear interconnected systems have got some important results, such as in related works [33][34][35][36][37][38][39][40][41][42] and references herein. Other works [33][34][35] have proposed a decentralized optimal stabilization control strategy for a class of continuous-time nonlinear interconnected large-scale systems. Qu et al 36 developed the state-feedback decentralized tracking problems for a class of nonlinear large-scale interconnected systems.…”

Adaptive‐dynamic‐programming‐based fuzzy control for triangular structure nonlinear uncertain systems with unknown time delay

“…It is well known that optimal control problems have received increasing attention and some valuable results have been developed (see other works [13][14][15][16][17][18][19][20][21][22][23] and the references therein). In traditional optimal control of nonlinear systems, the Hamilton-Jacobi-Bellman (HJB) equation usually needs to be solved, but obtaining the solution of the HJB equation is challenging and difficult because a closed-form analytical method does not exist in it.…”

“…Then, Liu et al 22 extended the SISO nonlinear system to a nonlinear large-scale system and proposed a decentralized optimal control scheme. In addition, the control approaches in the aforementioned works [18][19][20][21][22] can be applied only to nonlinear systems with known functions. Although Yang et al 23 proposed an ADP-based optimal control method for nonlinear systems with unknown functions, this adaptive optimal control method limits the controlled nonlinear systems from satisfying the matching condition [18][19][20][21][22] and other optimal methods, eg, H ∞ control 24 and reinforcement learning.…”