Deep Reinforcement Learning for Electric Transmission Voltage Control

Abstract

Today, human operators primarily perform voltage control of the electric transmission system. As the complexity of the grid increases, so does its operation, suggesting additional automation could be beneficial. A subset of machine learning known as deep reinforcement learning (DRL) has recently shown promise in performing tasks typically performed by humans. This paper applies DRL to the transmission voltage control problem, presents open-source DRL environments for voltage control, proposes a novel modification to the "deep Q network" (DQN) algorithm, and performs experiments at scale with systems up to 500 buses. The promise of applying DRL to voltage control is demonstrated, though more research is needed to enable DRL-based techniques to consistently outperform conventional methods.

Rethink AI-based Power Grid Control: Diving Into Algorithm Design

“…The pioneer work in [4] presents a novel autonomous control paradigm called "Grid Mind" to derive fast and effective controls in real time with Deep Q Network (DQN) agent to eliminate voltage violations. Later on, the follow-up work in [5] expands the findings of [4] and modified the DQN algorithm to improve control performance by avoiding the choice of same actions multiple times and normalizing the observations. Similar ideas have been further expanded in [6] and [7] for controlling the voltage setpoint of generators and PV-converters in a continuous manner by employing Deep Deterministic Policy Gradient (DDPG) agent.…”