Double deep Q‐learning coordinated control of hybrid energy storage system in island micro‐grid

Abstract

Summary It is difficult for a single energy storage to meet both power and energy requirements in the island micro‐grid because of the randomness of wind and solar irradiation. A reasonable way is to use hybrid energy storage in the island micro‐grid. For the energy management and optimization control of energy storage systems, there are various problems with traditional methods, such as the large computational complexity in dynamic programming. Q‐learning has recently been applied to the optimal control of energy storage systems. Due to the limitations of the Q‐learning algorithm in the state space, this article uses the Double deep Q‐learning (DQN) algorithm to design the control strategy of energy storage systems. It is applied to an island Micro‐grid system consisting of photovoltaic (PV), wind turbine, hydrogen storage (long‐term energy storage devices), and battery (short‐term energy storage devices). Transform the coordinated control of the hybrid energy storage system into a sequence decision problem. Due to the influence of renewable energy, load and other factors, different control strategies have different effects. DDQN algorithm combines the perception ability of deep learning with the decision‐making ability of reinforcement learning which can realize real‐time online decision control after training. Experimental results show that, the method of this article can be effectively processed for different weather scenarios and increase utilization of renewable energy.

Low‐carbon economic dispatch of the combined heat and power‐virtual power plants: A improved deep reinforcement learning‐based approach

“…The DRL-based algorithms have been applied to energy dispatch problems in various fields such as microgrids, CHP systems, and hybrid electric vehicles. The first DRL-based algorithms to be applied are deep Q-network (DQN) [43] and its variants, such as double DQN [44], prioritized experience replay DQN [45], and duelling DQN [46]. Sanaye et al developed a deterministic DQN algorithm to determine the operational strategy for the hybrid system without online trial and error or historical data on system performance [43].…”

“…The DRL‐based algorithms have been applied to energy dispatch problems in various fields such as microgrids, CHP systems, and hybrid electric vehicles. The first DRL‐based algorithms to be applied are deep Q‐network (DQN) [43] and its variants, such as double DQN [44], prioritized experience replay DQN [45], and duelling DQN [46]. Sanaye et al.…”

Deep Reinforcement Learning for Control of Microgrids: A Review

“…Quantum learning combined with DRL is applied to optimize real-time control in electrical systems featuring high penetration of alterna tive energy sources [54]. A double-deep Q-learning technique has been applied to an islanded microgrid with energy storage capabilities for cooperative control and energy management in different weather conditions [55]. A summary of this section's research is presented in table 3.…”

Data-Driven Self-Learning Controller Design Approach for Power-Aware IoT Devices based on Double Q-Learning Strategy

“…However, traditional methods are associated with certain problems, for example, large computational complexity in dynamic programming [15], [16]. Modifications to conventional algorithms have been established to eliminate these limitations.…”