Modeling and energy dynamic control for a ZEH via hybrid model-based deep reinforcement learning

Li, Yanxue; Wang, Zixuan; Xu, Wenya; Gao, Weijun; Xu, Yang; Xiao, Fu

doi:10.1016/j.energy.2023.127627

Cited by 17 publications

(3 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A key component of the distributed double deep Q-network (D3QN) algorithm also includes the advantage function, which considers the value and uncertainty of each action. However, in terms of algorithm strategy, the D3QN algorithm still adopts the ε-greedy strategy, which is to choose the action with the highest Q-value in the current state [38]. Figure 3 illustrates the pathway process of AFUCB-DQN.…”

Section: Algorithm Flowmentioning

confidence: 99%

Optimization Control Strategy for a Central Air Conditioning System Based on AFUCB-DQN

et al. 2023

View full text Add to dashboard Cite

The central air conditioning system accounts for 50% of the building energy consumption, and the cold source system accounts for more than 60% of the total energy consumption of the central air conditioning system. Therefore, it is crucial to solve the optimal control strategy of the cold source system according to the cooling load demand, and adjust the operating parameters in time to achieve low energy consumption and high efficiency. Due to the complex and changeable characteristics of the central air conditioning system, it is often difficult to achieve ideal results using traditional control methods. In order to solve this problem, this study first coupled the building cooling load simulation environment and the cold source system simulation environment to build a central air conditioning system simulation environment. Secondly, noise interference was introduced to reduce the gap between the simulated environment and the actual environment, and improve the robustness of the environment. Finally, combined with deep reinforcement learning, an optimal control strategy for the central air conditioning system is proposed. Aiming at the simulation environment of the central air conditioning system, a new model-free algorithm is proposed, called the dominant function upper confidence bound deep Q-network (AFUCB-DQN). The algorithm combines the advantages of an advantage function and an upper confidence bound algorithm to balance the relationship between exploration and exploitation, so as to achieve a better control strategy search. Compared with the traditional deep Q-network (DQN) algorithm, double deep Q-network (DDQN) algorithm, and the distributed double deep Q-network (D3QN) algorithm, the AFUCB-DQN algorithm has more stable convergence, faster convergence speed, and higher reward. In this study, significant energy savings of 21.5%, 21.4%, and 22.3% were obtained by conducting experiments at indoor thermal comfort levels of 24 °C, 25 °C, and 26 °C in the summer.

show abstract

Section: Algorithm Flowmentioning

confidence: 99%

Optimization Control Strategy for a Central Air Conditioning System Based on AFUCB-DQN

et al. 2023

View full text Add to dashboard Cite

show abstract

“…This indicates that the tw MAPPO method proposed in this paper exhibits superior optimization perform addressing microgrid scheduling problems. The two-stage MAPPO method proposed in this paper is compared to other deep reinforcement learning algorithms applied to the microgrid scheduling problem in discrete spaces, including the PPO algorithm [35], DQN algorithm [36], dueling deep Q-network (DDQN) algorithm [37], dueling double deep Q-network (D3QN) algorithm [38], advantage actor-critic (A2C) algorithm [39], and asynchronous advantage actor-critic (A3C) algorithm [40]. These algorithms are each applied to the microgrid model designed in this paper for training for 800 episodes.…”

Section: Comparative Analysismentioning

confidence: 99%

A Novel Two-Stage, Dual-Layer Distributed Optimization Operational Approach for Microgrids with Electric Vehicles

Zhou,

Zhang,

et al. 2023

Mathematics

View full text Add to dashboard Cite

As the ownership of electric vehicles (EVs) continues to rise, EVs are becoming an integral part of urban microgrids. Incorporating the charging and discharging processes of EVs into the microgrid’s optimization scheduling process can serve to load leveling, reducing the reliance of the microgrid on external power networks. This paper proposes a novel two-stage, dual-layer distributed optimization operational approach for microgrids with EVs. The lower layer is a distributed control layer, which ensures, through consensus control methods, that every EV maintains a consistent charging/discharging and state of charge (SOC). The upper layer is the optimization scheduling layer, determining the optimal operational strategy of the microgrid using the multiagent reinforcement learning method and providing control reference signals for the lower layer. Additionally, this paper categorizes the charging process of EVs into two stages based on their SOC: the constrained scheduling stage and the free scheduling stage. By employing distinct control methods during these two stages, we ensure that EVs can participate in the microgrid scheduling while fully respecting the charging interests of the EV owners.

show abstract

“…In the US, buildings consume 75% of the total national electricity, and building energy consumption contributed to 22% of national carbon emissions in China in 2021 [2]. In the context of buildings' high energy consumption, it is essential to promote the development of distributed renewable energy sources to decarbonize building energy systems [3,4]. Conventional buildings only rely on importing electricity from the public grid, while zero-energy houses (ZEHs) can use on-site energy sources as well as grid imports to meet the energy demand [5].…”

Section: Introductionmentioning

confidence: 99%

Performance Assessment and Comparative Analysis of Photovoltaic-Battery System Scheduling in an Existing Zero-Energy House Based on Reinforcement Learning Control

et al. 2023

Energies

Self Cite

View full text Add to dashboard Cite

The development of distributed renewable energy resources and smart energy management are efficient approaches to decarbonizing building energy systems. Reinforcement learning (RL) is a data-driven control algorithm that trains a large amount of data to learn control policy. However, this learning process generally presents low learning efficiency using real-world stochastic data. To address this challenge, this study proposes a model-based RL approach to optimize the operation of existing zero-energy houses considering PV generation consumption and energy costs. The model-based approach takes advantage of the inner understanding of the system dynamics; this knowledge improves the learning efficiency. A reward function is designed considering the physical constraints of battery storage, photovoltaic (PV) production feed-in profit, and energy cost. Measured data of a zero-energy house are used to train and test the proposed RL agent control, including Q-learning, deep Q network (DQN), and deep deterministic policy gradient (DDPG) agents. The results show that the proposed RL agents can achieve fast convergence during the training process. In comparison with the rule-based strategy, test cases verify the cost-effectiveness performances of proposed RL approaches in scheduling operations of the hybrid energy system under different scenarios. The comparative analysis of test periods shows that the DQN agent presents better energy cost-saving performances than Q-learning while the Q-learning agent presents more flexible action control of the battery with the fluctuation of real-time electricity prices. The DDPG algorithm can achieve the highest PV self-consumption ratio, 49.4%, and the self-sufficiency ratio reaches 36.7%. The DDPG algorithm outperforms rule-based operation by 7.2% for energy cost during test periods.

show abstract

Modeling and energy dynamic control for a ZEH via hybrid model-based deep reinforcement learning

Cited by 17 publications

References 22 publications

Optimization Control Strategy for a Central Air Conditioning System Based on AFUCB-DQN

Optimization Control Strategy for a Central Air Conditioning System Based on AFUCB-DQN

A Novel Two-Stage, Dual-Layer Distributed Optimization Operational Approach for Microgrids with Electric Vehicles

Performance Assessment and Comparative Analysis of Photovoltaic-Battery System Scheduling in an Existing Zero-Energy House Based on Reinforcement Learning Control

Contact Info

Product

Resources

About