With the growing integration of distributed energy resources (DERs), flexible loads, and other emerging technologies, there are increasing complexities and uncertainties for modern power and energy systems. This brings great challenges to the operation and control. Besides, with the deployment of advanced sensor and smart meters, a large number of data are generated, which brings opportunities for novel data-driven methods to deal with complicated operation and control issues. Among them, reinforcement learning (RL) is one of the most widely promoted methods for control and optimization problems. This paper provides a comprehensive literature review of RL in terms of basic ideas, various types of algorithms, and their applications in power and energy systems. The challenges and further works are also discussed.
Recent studies have shown that due to the hammer effect of the governor, hydropower units are easily creating negative damping torque at the common mode frequency (below 0.1 Hz). Therefore, there is a risk of ultra low frequency oscillations (ULFO) in hydropowerdominated systems. ULFO is a small-signal frequency oscillation problem, which is quite different from low frequency oscillations (LFO). A conventional power system stabilizer (CPSS) has less effect on suppressing ULFO. To solve this problem, this paper proposes a high-order polynomial structure to replace the CPSS, and combine it with a proportional resonance controller to form a novel PR-PSS. In order to ensure the robustness of PR-PSS, based on the characteristic analysis results of the PR-PSS, a deep reinforcement learning (DRL) algorithm asynchronous advantage actor-critic (A3C) is introduced to train an agent. After training, the proposed agent can provide optimal parameter settings for PR-PSS under various operating conditions. Simulation results verify the effectiveness of the proposed method.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.