Deep Q‐network application for optimal energy management in a grid‐tied solar PV‐Battery microgrid

Abstract

This paper presents a deep Q‐network (DQN) technique to optimally manage energy resources in a microgrid in which the algorithm learns tasks in the same way as humans do. Every move the agent makes in the environment generates feedback which then motivate the agent to learn more about the environment and perform far more intelligent steps later in its learning stages. This paper proposes a DQN‐based energy management system that learns system uncertainties, including load demand, grid prices and volatile renewable power supply to ensure that energy is optimally dispatched in such a setting. The method uses experience replay and target network to increase learning speed and improve stability in previous research. The performance of the system is evaluated with slow, medium, and fast fluctuating load profiles and their combinations and also with different sets of actions. The system is designed to minimize both grid transactional costs and battery degradation costs. Results shows that the proposed system successfully learns from experience to raise the battery state of charge and optimally shift loads from a one‐time instance, reducing aggregate peak load and cost of energy purchased from the utility grid while maximizing revenue from energy exported to the grid.

Digital twin‐based online resilience scheduling for microgrids: An approach combining imitative learning and deep reinforcement learning

“…Here, the operation constraints of the ESS system include the charging and discharging power constraint (16)(17)(18)(19), time period coupling constraint (20), battery capacity constraint (21), and inverter capacity constraint (22).…”

“…DRL has both strong situational awareness and decision‐making ability and learns how to maximize rewards through continuous interaction with the environment in a trial‐and‐error manner, which provides solutions for complex physical control tasks [17]. In recent years, applications of DRL in MGs keep emerging, among which the commonly used algorithms are deep Q network [18] and its variants, such as multi‐agent deep Q network [19], double deep Q network [20], and deep deterministic policy gradient (DDPG) [21]. Though effective to some extent, DRL generally suffers from low data efficiency and time‐consuming training [22].…”