Optimizing the Post-Disaster Control of Islanded Microgrid: A Multi-Agent Deep Reinforcement Learning Approach

Abstract

Extreme disasters may cause the power supply to the distribution system (DS) to be interrupted. The DS is forced to operate in island mode and forms an islanded microgrid (MG). In order to improve the post-disaster resilience of the DS and to provide longer power supply for as many loads as possible with limited generation resources, this paper proposes a multi-agent deep reinforcement learning (DRL) method which realizes a dual control on the source and load sides of the MG. The problem of resilience improvement is converted to a sequential decision making problem, where the objective is to maximize the cumulative MG utility value over the power outage duration. A multi-agent DRL model is proposed to solve the sequential decision making problem. A dual control policy including energy storage management and load shedding strategy is put forward to maximize the utility value of the MG. A reinforcement learning (RL) environment based on OpenAI and OpenDSS for islanded MG is constructed as a simulator, which has a general interface compatible with, and also can be published to, OpenAI Gym. Numerical simulations are performed for an MG equipped with wind turbines, diesel generators, and storage devices to validate the effectiveness of the proposed method. The influences of available generation resources and power outage duration on the control policy are discussed, which validates the strong adaptability of the proposed method in different conditions.

A review of power system resilience assessment and enhancement approaches by focusing on wind farms and wind turbines

“…Whereas, no comprehensive definition for resilience has been provided [2]. In the investigated papers in this article, some adopted definitions are as follows: ‐the ability of the system to withstand against HILP [13–15] ‐the ability of the system to withstand against HILP events and quickly restore customer service from the extended outages [16–19] ‐the ability to prepare adequate and comprehensive respond and recover rapidly from major disruptions due to extreme events [20] ‐the capability of an object to fully recover to its original state after severe disruptions [21] ‐the ability to withstand and reduce the magnitude and duration of disruptive events, which includes the capability to anticipate, absorb, adapt, and rapidly recover from such event [22, 23] ‐the ability of a power grid to withstand against severe disturbances and mitigate the damaging impacts of such catastrophes [24] ‐the grid's ability to withstand extraordinary and HILP events that may have never been experienced before, and rapidly recovering from such disruptive events, and adapting its operation and structure to prevent or mitigate the impact of similar events in the future [25] …”

“…• an advanced model predictive control (MPC) to control the distributed energy resources (DER), minimize the impact of transient disruptions and speed up the response, and recovery time of the system [74] • a control between demand and supply by using ramp rate data of wind [48] • coordinated control between DER to manage of the demand and supply sides by the contribution of distribution system operator [82] • an optimizing method to control the islanded MG based on the multi-agent deep reinforcement learning by management of RES and load curtailment [17] • a bi-level method based on contingencies of the active distribution network against the wind storm, 1 st level: minimize the total cost, and 2 nd level: extracts the worst-case realization of the uncertainties to quickly start micro turbine (MT) and ESS after windstorm [13] • a controlling scheme of power flow according to demand and supply management to have a coordinated control in a DC highway [83] • coordinated control of resources and hourly network reconfiguration [81] • post-catastrophe system reconfiguration for distributed system [21] • CVR to control the power demand [24] • using DGs in islanded MG to keep the critical loads alive [84] • Coordination of WT allocation and network reconfiguration improve the performance of WT in islanded mode [80] • prioritizing the recovery of critical loads through the consideration of customer interruption cost [71] Planning Before an event, PA…”

Exploiting Battery Storages With Reinforcement Learning: A Review for Energy Professionals

“…Energy harvesting user equipment [64] Optimal power control policy Not relevant Not considered Large energy harvesting networks [65] Real-time energy management of hybrid storage Not relevant Min/max limits for SoC Wave energy conversion system with hybrid storage [66] Optimal policy allocation for ensuring quality of data transmission Not relevant Min/Max battery capacity Energy harvesting in underwater relay network [67] Optimize online policy for wireless energy transfer Not relevant Not Considered Energy harvesting RF-powered communication systems [68] Graceful degradation when grid connection is lost Not relevant Not considered Islanded microgrid [69] Maximize PV self-consumption Not affected Charge/discharge efficiencies House with PV, buffered heat pump & battery [70] Select between power sources to minimize consumption (hydrogen equivalent) Reduce energy consumption of the UAV batteries Not relevant Limits for battery threshold Unmanned aerial vehicle network in the presence of jammer [106] Minimize the overall data packet loss Not relevant Battery levels are considered.…”

“…Phan & Lai [169] and Zhang et al [96] note that the trend towards a decentralized electric power system should in some seashore regions be complemented with a move to decentralized freshwater production, so a desalination plant is added to the microgrid. Nie et al [68] curtail loads to keep the microgrid operational for as long as possible.…”

Federated Multiagent Deep Reinforcement Learning Approach via Physics-Informed Reward for Multimicrogrid Energy Management

“…Besides, a MADRL method is adopted in Ref. [21] to realize the post-disaster resilience of distributed MG system. Aiming to increase the income of the system, the MADRL shows its strong adaptability in different conditions through experiments.…”