Deep reinforcement learning for power system: An overview

Zhang, Zidong; Zhang, Dongxia; Qiu, Robert C.

doi:10.17775/cseejpes.2019.00920

Cited by 137 publications

(22 citation statements)

References 76 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With reference to power systems domains, RL approaches are a well-studied topic, with RL being applied to a multitude of problems from various aspects [Zhang et al, 2020a, Glavic et al, 2017. In the context of VVC, existing studies were mainly focused on various aspects of scaling RL to the challenges specific to the VVC, such as minimizing constraint violations [Wang et al, 2020b] and scaling to combinatorially large actions spaces [Zhang et al, 2021].…”

Section: Related Workmentioning

confidence: 99%

A Graph Policy Network Approach for Volt-Var Control in Power Distribution Systems

Lee¹,

Sarkar²,

Wang³

2021

Preprint

View full text Add to dashboard Cite

Volt-var control (VVC) is the problem of operating power distribution systems within healthy regimes by controlling actuators in power systems. Existing works have mostly adopted the conventional routine of representing the power systems (a graph with tree topology) as vectors to train deep reinforcement learning (RL) policies. We propose a framework that combines RL with graph neural networks and study the benefits and limitations of graph-based policy in the VVC setting. Our results show that graph-based policies converge to the same rewards asymptotically however at a slower rate when compared to vector representation counterpart. We conduct further analysis on the impact of both observations and actions: on the observation end, we examine the robustness of graph-based policy on two typical data acquisition errors in power systems, namely sensor communication failure and measurement misalignment. On the action end, we show that actuators have various impacts on the system, thus using a graph representation induced by power systems topology may not be the optimal choice. In the end, we conduct a case study to demonstrate that the choice of readout function architecture and graph augmentation can further improve training performance and robustness.

show abstract

Section: Related Workmentioning

confidence: 99%

A Graph Policy Network Approach for Volt-Var Control in Power Distribution Systems

Lee¹,

Sarkar²,

Wang³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The application of RL to control and manage various aspects of power systems is a well-studied topic in literature [Zhang et al, 2020]. In the past few years, there has been renewed interest in this topic due to algorithmic advancements, allowing RL to go beyond tabular settings and scale to large state and action spaces using neural networks as expressive function approximators.…”

Section: Related Workmentioning

confidence: 99%

PowerGym: A Reinforcement Learning Environment for Volt-Var Control in Power Distribution Systems

Fan¹,

Lee²,

Wang³

2021

Preprint

View full text Add to dashboard Cite

We introduce PowerGym, an open-source reinforcement learning environment for Volt-Var control in power distribution systems. Following OpenAI Gym APIs, PowerGym targets minimizing power loss and voltage violations under physical networked constraints. PowerGym provides four distribution systems (13Bus, 34Bus, 123Bus, and 8500Node) based on IEEE benchmark systems and design variants for various control difficulties. To foster generalization, PowerGym offers a detailed customization guide for users working with their distribution systems. As a demonstration, we examine state-of-the-art reinforcement learning algorithms in PowerGym and validate the environment by studying controller behaviors. The repository is available at https://github.com/siemens/powergym.

show abstract

“…Other domains where RL has been used include hospital decision-making [31], precision agriculture [32], and fluid mechanics [33]. It is of little surprise that RL has been extensively used to solve various problems arising in energy systems [34], [35], [36], [37], [38]. Another review article on the use of RL [38] considers three application areas frequency and voltage control as well as energy management.…”

Section: Introductionmentioning

confidence: 99%

Reinforcement Learning: Theory and Applications in HEMS

Alani¹,

Das²

2022

Preprint

View full text Add to dashboard Cite

The twin capabilities of learning from experience and learning at higher levels of abstraction, set reinforcement learning apart from other areas of machine learning and (within the broader context) all of artificial intelligence. It allows algorithmic agents to replace human beings in the real world, including in homes and buildings, in application domains that had hitherto been considered to be beyond today’s capabilities. This goal, specifically aimed at home energy automation that forms the backdrop of this article, which surveys the use of deep reinforcement learning in various HEMS applications. The article provides an overview of generic reinforcement learning. This is followed with discussions on the state-of-the-art methods for value based, policy gradient, and actor-critic methods in deep reinforcement learning. In order to make published literature in reinforcement learning more accessible to HEMS researchers, verbal descriptions are accompanied with explanatory figures as well as mathematical expressions using the same terminology as the machine learning community. Next, a detailed survey of how reinforcement learning is used in different HEMS domains is described. The survey also considers what kind of reinforcement learning algorithms are used in each HEMS application. The survey suggests that this research is still in its infancy.

show abstract

Deep reinforcement learning for power system: An overview

Cited by 137 publications

References 76 publications

A Graph Policy Network Approach for Volt-Var Control in Power Distribution Systems

A Graph Policy Network Approach for Volt-Var Control in Power Distribution Systems

PowerGym: A Reinforcement Learning Environment for Volt-Var Control in Power Distribution Systems

Reinforcement Learning: Theory and Applications in HEMS

Contact Info

Product

Resources

About