Energy storage plays a significant role in improving the stability of distributed energy, improving power quality and peak regulation in the micro-grid system, which is of great significance to the sustainable development of energy. In grid-connected mode, energy storage is mainly used to reduce the operating costs of micro-grid. Real-time price arbitrage is an important source of energy storage revenue. It is feasible to design arbitrage strategies using Q-learning algorithm. Due to the overestimation of the Q learning algorithm, this paper proposes an arbitrage strategy method based on Double-Q learning. Compared with Q-learning algorithm, Double-Q learning can avoid overestimation and provide more stable and accurate arbitrage strategy for energy storage systems. Since the source of arbitrage in previous studies was limited to electricity prices alone, this paper considers joint arbitrage of electricity and carbon prices. The simulation results show that if adding fluctuate carbon prices to arbitrage sources, the arbitrage profits will increase by more than 110%. INDEX TERMS Energy storage, micro-grid system, double-Q learning, carbon prices.
Summary It is difficult for a single energy storage to meet both power and energy requirements in the island micro‐grid because of the randomness of wind and solar irradiation. A reasonable way is to use hybrid energy storage in the island micro‐grid. For the energy management and optimization control of energy storage systems, there are various problems with traditional methods, such as the large computational complexity in dynamic programming. Q‐learning has recently been applied to the optimal control of energy storage systems. Due to the limitations of the Q‐learning algorithm in the state space, this article uses the Double deep Q‐learning (DQN) algorithm to design the control strategy of energy storage systems. It is applied to an island Micro‐grid system consisting of photovoltaic (PV), wind turbine, hydrogen storage (long‐term energy storage devices), and battery (short‐term energy storage devices). Transform the coordinated control of the hybrid energy storage system into a sequence decision problem. Due to the influence of renewable energy, load and other factors, different control strategies have different effects. DDQN algorithm combines the perception ability of deep learning with the decision‐making ability of reinforcement learning which can realize real‐time online decision control after training. Experimental results show that, the method of this article can be effectively processed for different weather scenarios and increase utilization of renewable energy.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
customersupport@researchsolutions.com
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.