A Data-driven Method for Fast AC Optimal Power Flow Solutions via Deep Reinforcement Learning

Zhou, Yuhao; Zhang, Bei; Xu, Chunlei; Lan, Tian-Syung; Diao, Ruisheng; Shi, Di; Wang, Zhiwei; Lee, Wei-Jen

doi:10.35833/mpce.2020.000522

Cited by 65 publications

(19 citation statements)

References 29 publications

(59 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other related works explore formal guarantees for neural networks when learning OPF problems [26], [27], and extend the learning methodologies to security-constrained OPF problems [28], [29]. Reinforcement-learning approaches for OPF problems have also been proposed (e.g., [30]- [33]) and primarily focus on tackling real-time issues. This paper continues the line of work [6], [11], [34], [35] in using deep learning to predict AC-OPF solutions directly, while integrating practices/constraints found in U.S. energy markets.…”

Section: Related Workmentioning

confidence: 99%

Learning Regionally Decentralized AC Optimal Power Flows with ADMM

Mak¹,

Chatzos²,

Tanneau³

et al. 2022

Preprint

View full text Add to dashboard Cite

One potential future for the next generation of smart grids is the use of decentralized optimization algorithms and secured communications for coordinating renewable generation (e.g., wind/solar), dispatchable devices (e.g., coal/gas/nuclear generations), demand response, battery & storage facilities, and topology optimization. The Alternating Direction Method of Multipliers (ADMM) has been widely used in the community to address such decentralized optimization problems and, in particular, the AC Optimal Power Flow (AC-OPF). This paper studies how machine learning may help in speeding up the convergence of ADMM for solving AC-OPF. It proposes a novel decentralized machine-learning approach, namely ML-ADMM, where each agent uses deep learning to learn the consensus parameters on the coupling branches. The paper also explores the idea of learning only from ADMM runs that exhibit high-quality convergence properties, and proposes filtering mechanisms to select these runs. Experimental results on test cases based on the French system demonstrate the potential of the approach in speeding up the convergence of ADMM significantly.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning Regionally Decentralized AC Optimal Power Flows with ADMM

Mak¹,

Chatzos²,

Tanneau³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…2 (10) where θ t = r+γ max a Q(s , a; ω t−1 ) is the target for iteration t. Equation ( 10) samples the environment and stores the observed experiences in a replay memory, then a small batch is selected for learning using a gradient descent update step.…”

Section: Dqn-per-based Energy Trading Strategymentioning

confidence: 99%

“…Meanwhile, the recent advancement in artificial intelligence, mainly reinforcement learning (RL) has emerged for (near) optimal control of dynamic systems in energy network including energy trading [9], [10]. This is achieved by transforming the energy control or trading problem into a sequential decisionmaking problem and using RL to solve it.…”

Section: Introductionmentioning

confidence: 99%

Trading Strategy in a Local Energy Market, a Deep Reinforcement Learning Approach

Jogunola

Tsado

Adebisi

et al. 2021

2021 IEEE Electrical Power and Energy Conference (EPEC)

View full text Add to dashboard Cite

In response to energy transition fueled by the increasing energy generation mix and dynamic environment, this paper presents an energy trading strategy utilising real microgrid data. Specifically, we adapted the deep Q-network (DQN) with prioritised experience replay (PER) to develop a DQN-PERbased energy market algorithm to optimise the utility derived by prosumers participating in a local energy market (LEM). The problem of exercising energy trading actions is formulated as a sequential decision-making problem to optimise the prosumer's utility in a variety of energy trading scenarios. This includes the contingency or flexibility provided by the energy storage system (ESS), the incorporation of solar photovoltaic (PV) sources and the decision to trade energy with the grid or in a LEM. The results show the benefit achieved in trading energy in LEM with higher benefits when more sources of renewable energy are incorporated. For instance, the average benefit of trading in the LEM over the grid with ESS is 35%, which increased to 54% when PV and ESS are incorporated.

show abstract

“…Deep reinforcement learning (DRL) is a branch of machine learning algorithms and an important method of stochastic control based on the Markov decision process, which can better solve sequential decision problems (Sutton and Barto, 1998). Recently, DRL has been successfully implemented on many applications of power systems, such as optimal power flow (Zhang et al, 2021a), demand response (Wen et al, 2015), energy management system for microgrid (Venayagamoorthy et al, 2016), autonomous voltage control (Zhang et al, 2016b), and AGC (Zhou et al, 2020;Xi et al, 2021). In AGC problems, as stated previously, the presented literatures usually focus on the power allocation problem which still belongs to the conventional AGC strategy.…”

Section: Introductionmentioning

confidence: 99%

An AGC Dynamic Optimization Method Based on Proximal Policy Optimization

Liu

Li²,

Zhang

et al. 2022

Front. Energy Res.

View full text Add to dashboard Cite

The increasing penetration of renewable energy introduces more uncertainties and creates more fluctuations in power systems than ever before, which brings great challenges for automatic generation control (AGC). It is necessary for grid operators to develop an advanced AGC strategy to handle fluctuations and uncertainties. AGC dynamic optimization is a sequential decision problem that can be formulated as a discrete-time Markov decision process. Therefore, this article proposes a novel framework based on proximal policy optimization (PPO) reinforcement learning algorithm to optimize power regulation among each AGC generator in advance. Then, the detailed modeling process of reward functions and state and action space designing is presented. The application of the proposed PPO-based AGC dynamic optimization framework is simulated on a modified IEEE 39-bus system and compared with the classical proportional−integral (PI) control strategy and other reinforcement learning algorithms. The results of the case study show that the framework proposed in this article can make the frequency characteristic better satisfy the control performance standard (CPS) under the scenario of large fluctuations in power systems.

show abstract

A Data-driven Method for Fast AC Optimal Power Flow Solutions via Deep Reinforcement Learning

Cited by 65 publications

References 29 publications

Learning Regionally Decentralized AC Optimal Power Flows with ADMM

Learning Regionally Decentralized AC Optimal Power Flows with ADMM

Trading Strategy in a Local Energy Market, a Deep Reinforcement Learning Approach

An AGC Dynamic Optimization Method Based on Proximal Policy Optimization

Contact Info

Product

Resources

About