A deep asynchronous actor‐critic learning‐based event‐triggered decentralized load frequency control of power systems with communication delays

Abstract

This article proposes a novel asynchronous advantage actor‐critic (A3C) learning‐based dynamic event‐triggered mechanism for the decentralized load frequency regulation to alleviate the local‐area communication burden and influence of the load fluctuations. The proposed dynamic event‐triggered mechanism applies the A3C algorithm to optimally adjust the threshold of the event‐triggered function in real time. In the A3C algorithm framework, the long short‐term memory (LSTM) network is used to estimate the policy function and value function. First, for each control area, a novel model of the decentralized load frequency control (LFC) system is established to design the event‐triggered communication mechanism and deal with the communication delay simultaneously. Then, based on the Lyapunov stability theory, the controller gain parameters of the decentralized LFC system and the margins of the even‐triggering thresholds are derived by solving a series of linear matrix inequalities (LMIs). Finally, a three‐area and four‐area power systems are used to evaluate the proposed decentralized LFC method. Simulation results show that the proposed method can greatly reduce the data transmission times and preserve a satisfactory system performance.

Safe reward‐based deep reinforcement learning control for an electro‐hydraulic servo system

“…The flow chart of the safe reward-based DRL control method for the EHSS constrained control is shown in Figure 2. R cbf in (11) is designed as the safe reward for the DDPG in the proposed control method, to implement the safety constraints of e s min ≤ e s ≤ e s max in the EHSS control. Then the optimal safety policy 𝜋 * D is learned for the DDPG with R cbf in the proposed control method.…”

“…In order to improve the safety of the optimal policy learned in the DRL for the nonlinear system with safety constraints while accelerating the convergence speed, a safe reward R cbf is shaped by combining R(x, u, x ′ ) with a CBF-based potential difference term R Γ as shown in (11). R Γ is designed in the form of the potential function according to the convergence guarantee of the reward shaping method, 24,25 to speed up the convergence process of the DRL control method.…”

“…Remark 2. Any CBF that satisfies the safe properties of the CBF in Section 2.1 can be used in (11), and a classical CBF…”

“…Proposition 2 (Unbiased optimality). If R cbf is given in (11), then the optimal safety policy in the DRL with R cbf exists and is also safe and optimal in the original DRL with R(x, u, x ′ ).…”

“…10 Safety has become a critical problem in the design of the DRL architecture recently, especially when the DRL is used for the nonlinear control system with safety constraints under uncertainties and perturbations. 11,12 Without considering the safety, the policy learned in the DRL may lead the systems into dangerous scenes. For example, in the exoskeleton robot control for rehabilitation, 2 the policy without considering the safe moving range might result in the injury of the patients.…”