A Hybrid Multiagent Framework With Q-Learning for Power Grid Systems Restoration

Abstract

Abstract-This paper presents a hybrid multi-agent framework with a Q-learning algorithm to support rapid restoration of power grid systems following catastrophic disturbances involving loss of generators. This framework integrates the advantages of both centralised and decentralised architectures to achieve accurate decision making and quick responses when potential cascading failures are detected in power systems. By using this hybrid framework, which does not rely on a centralised controller, the single point of failure in power grid systems can be avoided. Further, the use of the Q-learning algorithm developed in conjunction with the restorative framework can help the agents to make accurate decisions to protect against cascading failures in a timely manner without requiring a global reward signal. Simulation results demonstrate the effectiveness of the proposed approach in comparison with the typical centralised and decentralised approaches based on several evaluation attributes.

Reinforcement Learning for Electric Power System Decision and Control: Past Considerations and Perspectives

“…Restorative control was considered in the past for transmission systems Ye et al, 2011. With the emergence of renewable energy generation the same needs exist for distribution systems and microgrids.…”

Power system restoration: a literature review from 2006 to 2016

“…Graph theory with Petri nets [9] is also employed, but verification of constraints and reduction of uncertainties both need improvement. Based on the regional distribution characteristics in space, multi-agent technologies [10][11][12][13] are developed with potential prospect. As a functional extension of expert systems and heuristic rules, decision support systems [14] have been demonstrated efficiently.…”

Decentralised dispatch of distributed energy resources in smart grids via multi-agent coalition formation

“…The agents in a multi-agent system can work autonomously, make decisions independently and interact with each other to achieve global goals. Multi-agent systems, as a new paradigm which can facilitate distributed control [31], have been adopted in power systems for various purposes, such as voltage support [5], power restoration [48] and system management [32].…”