Frequency regulation of off‐grid system with battery energy storage system using deep Q‐network

Abstract

This paper proposes a model-free decision algorithm for battery energy storage system (BESS) charging/discharging using deep reinforcement learning (DRL) to regulate off-grid frequency fluctuation. This method is novel since the frequency regulation problem is cast in an off-grid system to a deep Q-network framework, which avoids directly solving a specific optimization model or model-based control. The advantage of the proposed method is that the agents learn to comprehensively make short-term forecasts of demand and PV output, analyse trends in forecast errors, and create control signals within the framework of DRL. A reward function with two criteria, frequency target value and frequency constraint value, guides the agent to suppress the frequency fluctuation by adjusting the weight coefficient. The effectiveness of the proposed method is verified using computational simulations with the off-grid system model and actual power consumption data. The proposed method enables controlling the BESS to satisfy the balance of supply and demand on the off-grid, thus realizing a control method that does not require predictive information. Moreover, the frequency control can be achieved more accurately in a short computation time than the prediction-based method using the seasonal auto-regressive integrated moving average.