Reinforcement learning for building controls: The opportunities and challenges

Abstract

Building controls are becoming more important and complicated due to the dynamic and stochastic energy demand, on-site intermittent energy supply, as well as energy storage, making it difficult for them to be optimized by conventional control techniques. Reinforcement Learning (RL), as an emerging control technique, has attracted growing research interest and demonstrated its potential to enhance building performance while addressing some limitations of other advanced control techniques, such as model predictive control. This study conducted a comprehensive review of existing studies that applied RL for building controls. It provided a detailed breakdown of the existing RL studies that use a specific variation of each major component of the Reinforcement Learning: algorithm, state, action, reward, and environment. We found RL for building controls is still in the research stage with limited applications (11%) in real buildings. Three significant barriers prevent the adoption of RL controllers in actual building controls: (1) the training process is time consuming and data demanding, (2) the control security and robustness need to be enhanced, and (3) the generalization capabilities of RL controllers need to be improved using approaches such as transfer learning. Future research may focus on developing RL controllers that could be used in real buildings, addressing current RL challenges, such as accelerating training and enhancing control robustness, as well as developing an opensource testbed and dataset for performance benchmarking of RL controllers.

Deep Reinforcement Learning for Smart Home Energy Management

“…For example, the applications of DRL in power systems, communications and networking, autonomous IoT, cyber security, and multi-agent systems can be found in [20], [29]- [32]. In addition, there are several surveys on building energy systems, but the involved methods are RL [33]- [36] or other artificial intelligence methods (e.g., Model Predictive Control (MPC), Fuzzy Logic (FL)) [37]. Although some DRL algorithms are mentioned in [38] and [39], they mainly focus on different applications (ranging from load forecasting to cyber security) of RL/DRL in sustainable energy and electric systems.…”

Classification of retrofit measures for residential buildings according to the global cost

“…Cluster analysis is suitable for data preprocessing and often combined with supervised learning [46,47,48]. Reinforcement learning is a promising area for control, but its practical application is still limited [49,50,51].…”

Study on dynamic thermal characteristics of thermoelectric radiant cooling panel system through a hybrid method

“…At the same time, we noticed that machine learning method such as artificial neural network has been showing its power in solving some difficult problems that conventional is incapable or ineffective. The machine learning methods are found to be highly powerful for building energy prediction [ 32 , 33 ], and some researches starts using ANN for thermoelectric based system evaluation [ 34 ] or thermoelectric generator [ 35 ]. However, very few has shown its application for radiant cooling systems.…”