Evaluating the Adaptability of Reinforcement Learning Based HVAC Control for Residential Houses

Kurte, Kuldeep; Munk, Peter L.; Kotevska, Olivera; Amasyali, Kadir; Smith, Robert W.; McKee, Evan; Du, Yan; Cui, Borui; Kuruganti, Teja; Zandi, Helia

doi:10.3390/su12187727

Cited by 24 publications

(6 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Cicirelli et al [24] used DQN to balance energy consumption and thermal comfort. Kurte et al [25,26] applied DRL in residential HVAC control to save costs and maintain comfort.…”

Section: Related Workmentioning

confidence: 99%

MAQMC: Multi-Agent Deep Q-Network for Multi-Zone Residential HVAC Control

Ding¹,

Fu²,

Chen³

et al. 2023

Computer Modeling in Engineering &Amp; Sciences

View full text Add to dashboard Cite

The optimization of multi-zone residential heating, ventilation, and air conditioning (HVAC) control is not an easy task due to its complex dynamic thermal model and the uncertainty of occupant-driven cooling loads. Deep reinforcement learning (DRL) methods have recently been proposed to address the HVAC control problem. However, the application of single-agent DRL for multi-zone residential HVAC control may lead to non-convergence or slow convergence. In this paper, we propose MAQMC (Multi-Agent deep Q-network for multi-zone residential HVAC Control) to address this challenge with the goal of minimizing energy consumption while maintaining occupants' thermal comfort. MAQMC is divided into MAQMC2 (MAQMC with two agents:one agent controls the temperature of each zone, and the other agent controls the humidity of each zone) and MAQMC3 (MAQMC with three agents:three agents control the temperature and humidity of three zones, respectively). The experimental results show that MAQMC3 can reduce energy consumption by 6.27% and MAQMC2 by 3.73% compared with the xed point; compared with the rule-based, MAQMC3 and MAQMC2 respectively can reduce 61.89% and 59.07% comfort violation. In addition, experiments with di erent regional weather data demonstrate that the well-trained MAQMC RL agents have the robustness and adaptability to unknown environments.

show abstract

“…Cicirelli et al [24] used DQN to balance energy consumption and thermal comfort. Kurte et al [25,26] applied DRL in residential HVAC control to save costs and maintain comfort.…”

Section: Related Workmentioning

confidence: 99%

MAQMC: Multi-Agent Deep Q-Network for Multi-Zone Residential HVAC Control

Ding¹,

Fu²,

Chen³

et al. 2023

Computer Modeling in Engineering &Amp; Sciences

View full text Add to dashboard Cite

show abstract

“…These studies have already proved the feasibility and applicability of RL approaches to the various problems in the contexts of DR and BEMS. For example, [24][25][26] developed RL-based controllers to control HVAC systems. On the other hand, water heater control still remains largely untapped, except for only a few studies [17].…”

Section: Literature Reviewmentioning

confidence: 99%

Deep Reinforcement Learning for Autonomous Water Heater Control

et al. 2021

Self Cite

View full text Add to dashboard Cite

Electric water heaters represent 14% of the electricity consumption in residential buildings. An average household in the United States (U.S.) spends about USD 400–600 (0.45 ¢/L–0.68 ¢/L) on water heating every year. In this context, water heaters are often considered as a valuable asset for Demand Response (DR) and building energy management system (BEMS) applications. To this end, this study proposes a model-free deep reinforcement learning (RL) approach that aims to minimize the electricity cost of a water heater under a time-of-use (TOU) electricity pricing policy by only using standard DR commands. In this approach, a set of RL agents, with different look ahead periods, were trained using the deep Q-networks (DQN) algorithm and their performance was tested on an unseen pair of price and hot water usage profiles. The testing results showed that the RL agents can help save electricity cost in the range of 19% to 35% compared to the baseline operation without causing any discomfort to end users. Additionally, the RL agents outperformed rule-based and model predictive control (MPC)-based controllers and achieved comparable performance to optimization-based control.

show abstract

“…In addition to occupant related thermal factors, several conventional approaches reflecting spatial geometries, types of building envelopes, and motorized ventilation systems were developed to appropriately react extreme weather conditions. Meanwhile, comprehensive thermal comfort indices were gradually developed to approach specific requirements in respond to strengthened guidelines and regulations for local characteristics [20,21]. In many cases of modified approaches to effectively define meaningful interactions in the human factors, their inner structures of thermal systems and architectural components were investigated to compare mechanical functions with data-driven regression results for more reliable network-based models.…”

Section: Thermal Systems In Buildingsmentioning

confidence: 99%

Improvement of the Performance Balance between Thermal Comfort and Energy Use for a Building Space in the Mid-Spring Season

Ahn

2020

Sustainability

View full text Add to dashboard Cite

In thermal controls in buildings, recent statistical and data-driven approaches to optimize supply air conditions have been examined in association with several types of building spaces and patterns of energy consumption. However, many strategies may have some problems where high-control precision may increase energy use, or low energy use in systems may decrease indoor thermal quality. This study investigates a neural network algorithm with an adaptive model on how to control the supply air conditions reflecting learned data. During the process, the adaptive model complements the signals from the network to independently maintain the comfort level within setting ranges. Although the proposed model effectively optimizes energy consumption and supply air conditions, it achieves quite improved comfort levels about 14% more efficient than comparison models. Consequently, it is confirmed that a network and learning algorithm equipped with an adaptive controller properly responds to users’ comfort levels and system’s energy consumption in a single space. The improved performance in space levels can be significant in places where many spaces are systematically connected, and in places which require a high consistency of indoor thermal comfort. Another advantage of the proposed model is that it properly reduces an increase in energy consumption despite an intensive strategy is utilized to improve thermal comfort.

show abstract

Evaluating the Adaptability of Reinforcement Learning Based HVAC Control for Residential Houses

Cited by 24 publications

References 19 publications

MAQMC: Multi-Agent Deep Q-Network for Multi-Zone Residential HVAC Control

MAQMC: Multi-Agent Deep Q-Network for Multi-Zone Residential HVAC Control

Deep Reinforcement Learning for Autonomous Water Heater Control

Improvement of the Performance Balance between Thermal Comfort and Energy Use for a Building Space in the Mid-Spring Season

Contact Info

Product

Resources

About