Stability Constrained Reinforcement Learning for Real-Time Voltage Control

Shi, Yuanyuan; Qu, Guannan; Low, Steven H.; Anandkumar, Anima; Wierman, Adam

doi:10.48550/arxiv.2109.14854

Cited by 2 publications

(3 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, they are not suitable for safety-critical infrastructure. Two recent works [7,30] propose a model-free DRL approach for voltage control with stability guarantees. The main tool being used in [7,30] is Lyapunov stability theory, from which a structural constraint for stable controllers is derived, and policy optimization with the constraint is performed.…”

Section: Introductionmentioning

confidence: 99%

“…Two recent works [7,30] propose a model-free DRL approach for voltage control with stability guarantees. The main tool being used in [7,30] is Lyapunov stability theory, from which a structural constraint for stable controllers is derived, and policy optimization with the constraint is performed. In contrast, our framework jointly learns the system model (consistent to data) and stable controller, in an online fashion.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Robust online voltage control with an unknown grid topology

Yeh

Shi

et al. 2022

Proceedings of the Thirteenth ACM International Conference on Future Energy Systems

Self Cite

View full text Add to dashboard Cite

Voltage control generally requires accurate information about the grid's topology in order to guarantee network stability. However, accurate topology identification is a challenging problem for existing methods, especially as the grid is subject to increasingly frequent reconfiguration due to the adoption of renewable energy. Further, running existing control mechanisms with incorrect network information may lead to unstable control. In this work, we combine a nested convex body chasing algorithm with a robust predictive controller to achieve provably finite-time convergence to safe voltage limits in the online setting where the network topology is initially unknown. Specifically, the online controller does not know the true network topology and line parameters, but instead must learn them over time by narrowing down the set of network topologies and line parameters that are consistent with its observations and adjusting reactive power generation accordingly to keep voltages within desired safety limits. We demonstrate the effectiveness of the approach using a case study, which shows that in practical settings the controller is indeed able to narrow the set of consistent topologies quickly enough to make control decisions that ensure stability. CCS CONCEPTS• Hardware → Smart grid; Power networks; • Theory of computation → Online learning algorithms; • Computer systems organization → Sensor networks; • Computing methodologies → Control methods.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Robust online voltage control with an unknown grid topology

Yeh

Shi

et al. 2022

Proceedings of the Thirteenth ACM International Conference on Future Energy Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…Meanwhile, unbalanced power flow, multi-phase device integration, and the lack of accurate network models further complicate the situation. To this end, a number of studies [79]- [96] propose using model-free RL for voltage control. We summarize the related work in Table II and present below how to solve the voltage control problem in the RL framework.…”

Section: B Voltage Controlmentioning

confidence: 99%

Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges

Chen

Tang

et al. 2022

IEEE Trans. Smart Grid

Self Cite

153

View full text Add to dashboard Cite

With large-scale integration of renewable generation and distributed energy resources, modern power systems are confronted with new operational challenges, such as growing complexity, increasing uncertainty, and aggravating volatility. Meanwhile, more and more data are becoming available owing to the widespread deployment of smart meters, smart sensors, and upgraded communication networks. As a result, data-driven control techniques, especially reinforcement learning (RL), have attracted surging attention in recent years. This paper provides a comprehensive review of various RL techniques and how they can be applied to decision-making and control in power systems. In particular, we select three key applications, i.e., frequency regulation, voltage control, and energy management, as examples to illustrate RL-based models and solutions. We then present the critical issues in the application of RL, i.e., safety, robustness, scalability, and data. Several potential future directions are discussed as well.

show abstract

Stability Constrained Reinforcement Learning for Real-Time Voltage Control

Cited by 2 publications

References 20 publications

Robust online voltage control with an unknown grid topology

Robust online voltage control with an unknown grid topology

Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges

Contact Info

Product

Resources

About