Improved Deep Q-Network for User-Side Battery Energy Storage Charging and Discharging Strategy in Industrial Parks

Abstract

Battery energy storage technology is an important part of the industrial parks to ensure the stable power supply, and its rough charging and discharging mode is difficult to meet the application requirements of energy saving, emission reduction, cost reduction, and efficiency increase. As a classic method of deep reinforcement learning, the deep Q-network is widely used to solve the problem of user-side battery energy storage charging and discharging. In some scenarios, its performance has reached the level of human expert. However, the updating of storage priority in experience memory often lags behind updating of Q-network parameters. In response to the need for lean management of battery charging and discharging, this paper proposes an improved deep Q-network to update the priority of sequence samples and the training performance of deep neural network, which reduces the cost of charging and discharging action and energy consumption in the park. The proposed method considers factors such as real-time electricity price, battery status, and time. The energy consumption state, charging and discharging behavior, reward function, and neural network structure are designed to meet the flexible scheduling of charging and discharging strategies, and can finally realize the optimization of battery energy storage benefits. The proposed method can solve the problem of priority update lag, and improve the utilization efficiency and learning performance of the experience pool samples. The paper selects electricity price data from the United States and some regions of China for simulation experiments. Experimental results show that compared with the traditional algorithm, the proposed approach can achieve better performance in both electricity price systems, thereby greatly reducing the cost of battery energy storage and providing a stronger guarantee for the safe and stable operation of battery energy storage systems in industrial parks.

A Deep Reinforcement Learning Approach for Microgrid Energy Transmission Dispatching

“…In order to reduce the dependence on the model, the data-driven method is proposed in large quantities to solve the optimized dispatching problem of microgrids and improve the economic goals as much as possible under various constraints. Chen et al proposed a modified deep Q-network to solve the power-charging and -discharging problem of userside battery energy storage, reducing the cost and energy consumption of the charging and discharging of the industrial park [12]. Paudyal et al transformed the mixed integer nonlinear programming problem into a nonlinear programming problem based on actual needs and established a three-phase power distribution optimization power flow model, thereby reducing the computational burden and facilitating the Hierarchical implementation and application [13].…”

“…The voltage V m of the microgrid is generally 220 V. At the same time, the line loss of the h-th transmission line at time t can be calculated based on Formulas (12) and (13).…”

A Word-Granular Adversarial Attacks Framework for Causal Event Extraction

“…It currently performs well on many decision-based problems. For example, in games or other fields, it has many applications [ 21 , 22 , 23 ]. At the same time, we believe that the application of reinforcement learning to NLP still has great potential.…”

Improved Deep Q-network for Mobile Charging Sequence Scheduling with the Minimal Mobile Charger Energy Capacity in Wireless Rechargeable Sensor Network

“…For example, we proposed a multistep adaptive dynamic programming algorithm, one type of reinforcement learning, for cooperative target tracking in energy harvesting WSNs to schedule sensors over an infinite horizon (Liu et al, 2020). We also proposed an improved deep Q-network approach for user-side battery energy storage charging and discharging strategy to reduce the costs and energy consumptions of charging and discharging actions (Chen et al, 2021). In (Chen et al, 2022), we proposed an approach that combined reinforcement learning and traditional optimization methods to address the problems of insufficient performance in terms of economic operation and efficient dispatching.…”

“…In our previous work (Liu et al, 2020;Chen et al, 2021Chen et al, , 2022Jiang et al, 2022), reinforcement learning methods have been successfully applied to the charging scheduling problem of microgrids, energy harvesting WSNs, and WRSNs. For example, we proposed a multistep adaptive dynamic programming algorithm, one type of reinforcement learning, for cooperative target tracking in energy harvesting WSNs to schedule sensors over an infinite horizon (Liu et al, 2020).…”