Multiobjective Reinforcement Learning-Based Intelligent Approach for Optimization of Activation Rules in Automatic Generation Control

Abstract

This paper proposes a novel hybrid intelligent approach to solve the dynamic optimization problem of activation rules for automatic generation control (AGC) based on multiobjective reinforcement learning (MORL) and small population-based particle swarm optimization (SPPSO). The activation rule for AGC is to dynamically allocate the AGC regulating commands among various AGC units, and subsequently, the secondary control reserve of those units can be activated. Therefore, the activation rule for AGC is vital to ensure the overall control performance of AGC schemes. In this paper, MORL is applied to provide a customized platform for interactive self-learning to maximize the long-run discounted reward, i.e., minimize the generation cost, regulating error, and emission from a long-term viewpoint. SPPSO is utilized to effectively and efficiently obtain the optimality of activation rule with a fast convergence speed to fulfill the real-time requirement of AGC activation. Furthermore, a novel analytic hierarchy processbased coordination factor is introduced to identify the optimum multi-objective tradeoff in various power system operation scenarios. At last, the validation of the proposed hybrid method has been demonstrated via comprehensive tests using practical data from the dispatch center of China Southern Power Grids.

(Deep) Reinforcement learning for electric power system control and related problems: A short review and perspectives

“…Automatic Generation Control (AGC) and Load-Frequency Control (LFC) were considered in [35,[37][38][39][41][42][43][44][45][46] (the objective is to keep frequency in a narrow range around nominal value, for example in Europe [49.8 − 50.2]Hz). AGC and LFC differs in that AGC includes LFC together with generation dispatch function for control of so called area control error that is a parameterized sum of frequency deviation and active power flows over so-called tie-lines (the lines connecting subsystems within a larger interconnection).…”

“…Work presented in [38] suggested single AGC controller based on multi-step Q(λ) method while [39] suggested the use of correlated equilibrium Q(λ) within a multi-agent setting (similar approach was proposed in [40] with the difference that correlated equilibrium Q-learning was proposed within a multi-agent framework for AGC). A multi-objective Qlearning was used to activate rules of AGC (consisting of dynamic allocation of the AGC regulating commands among various AGC units, and activation of the secondary control reserve of those units was considered in [41]. Reference [42] proposed a lifelong learning control scheme for AGC where the wind farms, photovoltaic stations, and electric vehicles are aggregated as a wide-area virtual power plant participating in AGC with other generation plants.…”

Multi-Objective Optimization for Coordinated Day-Ahead Scheduling Problem of Integrated Electricity-Natural Gas System With Microgrid

“…Heuristic methods have aroused intense interest due to their versatility, flexibility, and robustness in seeking the global optimum solution, such as GSO [11], PSO [12], DE [13], GSA [14], and etc. Attempts have been made to apply these algorithms, including machine-learning algorithms, to tackle large-scale and economically important optimisation problems, such as optimal power flow and economic dispatch in power systems [15].…”

Virtual Generation Alliance Automatic Generation Control Based on Deep Reinforcement Learning

“…A traditional AGC strategy has difficulty dealing with the strong random disturbance. In the strategy, the total AGC generation power command of system is generated and dispatched to units through the proportion integration (PI) controller and the proportional dispatch method [4].…”