Online Area Load Modeling in Power Systems Using Enhanced Reinforcement Learning

Abstract

Abstract:The accuracy of load modeling directly influences power system operation and control. Previous modeling studies have mainly concentrated on the loads connected to a single boundary bus, without thoroughly considering the static load characteristics of the voltage. To remedy this oversight, this paper proposes an accurate modeling approach for area loads with multiple boundary buses and ZIP loads (a combination of constant-impedance, constant-current and constant-power loads) based on Ward equivalence. Furthermore, to satisfy the requirements for real-time monitoring, the model parameters are identified in an online manner using an enhanced reinforcement learning (ERL) algorithm. Parallel tables of value functions are implemented in the ERL algorithm to improve its tracking performance. Three simulation cases are addressed, the first involving a single ZIP load and the second and third involving area loads in the IEEE 57-bus system and in a real 1209-bus power system in China, respectively. The results demonstrate that the ERL algorithm outperforms an existing reinforcement learning algorithm and the improved least-squares method in terms of convergence and the ability to track both step-changing and time-varying loads. Additionally, the results obtained on test cases confirm that the proposed area load model is more accurate than a previously introduced model.

A Hybrid End-to-End Control Strategy Combining Dueling Deep Q-network and PID for Transient Boost Control of a Diesel Engine with Variable Geometry Turbocharger and Cooled EGR

“…The theory of reinforcement learning, inspired by the psychology of behaviorism, focuses on online learning and tries to maintain a balance between exploration and exploitation [18]. Different from supervised learning and unsupervised learning, reinforcement learning does not require any pre-given data, but obtains learning information and updates model parameters by receiving rewards (feedback) from the environment for actions [19][20][21]. Deep reinforcement learning (DRL), which excels at solving a wide variety of Atari and board games, is an area of machine learning that combines the deep learning approach and reinforcement learning (RL) [22,23].…”

Two‐level area‐load modelling for OPF of power system using reinforcement learning

“…If the structure and components of a load area are known, Ward [2,3], REI [4] and Thevenin [5] approaches can be applied to simplify the topological structure. Obtaining detailed information is not always possible with the increasing scale of distribution networks, which prompts researchers to turn to the measurement-based modelling [6][7][8]. Measurement-based modelling, which merely needs to collect a small amount of measurement data using monitoring devices such as synchronised phasor measurement units (PMUs) to derive load characteristics is a popular method.…”

“…The equivalent model is dependent on the voltage magnitude, which can better represent the actual operation of the power system. Shang et al [8] adopted an equivalent model composed of several interconnected fictitious branches among boundary buses and multiple ZIP loads.…”

Optimal location of PEVCSs using MAS and ER approach

“…In this paper, the RLA is introduced for PET agents to make decisions whether to find passengers or to charge, as well as to determine the optimal direction to cruise under various situations. There are many different kinds of RLAs such as learning automata [21,22], Sarsa [23] and Q-learning [24,25]. Traditional Q-learning has been widely applied in power systems and has achieved good results compared with others [24,25].…”

“…There are many different kinds of RLAs such as learning automata [21,22], Sarsa [23] and Q-learning [24,25]. Traditional Q-learning has been widely applied in power systems and has achieved good results compared with others [24,25]. However, its disadvantage of the slow convergence rate has a negative influence on the acquisition of optimal strategies [26].…”