Suboptimal reduced control of unknown nonlinear singularly perturbed systems via reinforcement learning

Abstract

In this paper, a suboptimal reduced control method is proposed for a class of nonlinear singularly perturbed systems (SPSs) with unknown dynamics. By using singular perturbation theory, the original system is reduced to a reduced system, by which a policy iterative method is proposed to solve the corresponding reduced Hamilton–Jacobi–Bellman (HJB) equation with convergence guaranteed. A reinforcement learning (RL) algorithm is proposed to implement the policy iterative method without using any knowledge of the system dynamics. In the RL algorithm, the unmeasurable state of the virtual reduced system is reconstructed by the slow state measurements of the original system, the controller and cost function are approximated by actor‐critic neural networks (NNs) and the method of weighted residuals is utilized to update the NN weights. The influence introduced by state reconstruction error and NN function approximation on the convergence, suboptimality of the reduced controller and stability of the closed‐loop SPSs are rigorously analyzed. Finally, the effectiveness of our proposed method is illustrated by examples.

Adaptive Neural Partial State Tracking Control for Full-State-Constrained Uncertain Singularly Perturbed Nonlinear Systems and Its Applications to Electric Circuit

“…In this way, the state constraints (2) of the original systems are converted to the tracking error constraints described by (10). Considering Equation (7) and system (1), we have…”

“…The decentralized control method for multi-agent linear SPSs was provided in [6], where the synchronization problem of networks of linear SPSs was rewritten as stabilization of uncertain linear SPSs to reduce the associated energy and the communication load. The suboptimal control strategy for nonlinear SPSs with known fast-subsystems was investigated in [7], in which it was innovative to the suboptimal performance of the considered systems without having any knowledge of the system. An adaptive dynamic programming scheme is designed for linear SPSs, whose slow-subsystems are unknown [8].…”