Practical implementation and evaluation of deep reinforcement learning control for a radiant heating system

Zhang, Zhiang; Lam, Khee Poh

doi:10.1145/3276774.3276775

Cited by 77 publications

(69 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ahn and Park [21] highlighted the need of approaches such as transfer learning in applying the DQN to enhance their self-learning ability so that trained RL models (in simulated environment) can be utilized in real-world situations. At present, we found two research papers focusing on the practical implementation and deployment of the DRL-based HVAC control [22,25]. In this study, we focus on evaluating the ability of DRL-based HVAC control to provide cost savings when pre-trained on one building model and deployed on different house models with varying user comforts.…”

Section: Reinforcement Learning Based Hvac Controlmentioning

confidence: 99%

Evaluating the Adaptability of Reinforcement Learning Based HVAC Control for Residential Houses

et al. 2020

View full text Add to dashboard Cite

Intelligent Heating, Ventilation, and Air Conditioning (HVAC) control using deep reinforcement learning (DRL) has recently gained a lot of attention due to its ability to optimally control the complex behavior of the HVAC system. However, more exploration is needed on understanding the adaptability challenges that the DRL agent could face during the deployment phase. Using online learning for such applications is not realistic due to the long learning period and likely poor comfort control during the learning process. Alternatively, DRL can be pre-trained using a building model prior to deployment. However, developing an accurate building model for every house and deploying a pre-trained DRL model for HVAC control would not be cost-effective. In this study, we focus on evaluating the ability of DRL-based HVAC control to provide cost savings when pre-trained on one building model and deployed on different house models with varying user comforts. We observed around 30% of cost reduction by pre-trained model over baseline when validated in a simulation environment and achieved up to 21% cost reduction when deployed in the real house. This finding provides experimental evidence that the pre-trained DRL has the potential to adapt to different house environments and comfort settings.

show abstract

Section: Reinforcement Learning Based Hvac Controlmentioning

confidence: 99%

Evaluating the Adaptability of Reinforcement Learning Based HVAC Control for Residential Houses

et al. 2020

View full text Add to dashboard Cite

show abstract

“…DRL strategies have also been implemented in real buildings, although in small experiments. Chen et al [51] used DRL to control the damper position of a VAV box in a single conference room in a building and Zhang and Lam [52] used it to control the supply water setpoint to the HVAC system in a experimental test-bed office in a university building.…”

Section: Literature Review and Contributionmentioning

confidence: 99%

Solar+ Optimizer: A Model Predictive Control Optimization Platform for Grid Responsive Building Microgrids

et al. 2020

View full text Add to dashboard Cite

With the falling costs of solar arrays and battery storage and reduced reliability of the grid due to natural disasters, small-scale local generation and storage resources are beginning to proliferate. However, very few software options exist for integrated control of building loads, batteries and other distributed energy resources. The available software solutions on the market can force customers to adopt one particular ecosystem of products, thus limiting consumer choice, and are often incapable of operating independently of the grid during blackouts. In this paper, we present the “Solar+ Optimizer” (SPO), a control platform that provides demand flexibility, resiliency and reduced utility bills, built using open-source software. SPO employs Model Predictive Control (MPC) to produce real time optimal control strategies for the building loads and the distributed energy resources on site. SPO is designed to be vendor-agnostic, protocol-independent and resilient to loss of wide-area network connectivity. The software was evaluated in a real convenience store in northern California with on-site solar generation, battery storage and control of HVAC and commercial refrigeration loads. Preliminary tests showed price responsiveness of the building and cost savings of more than 10% in energy costs alone.

show abstract

“…A model-free Q-learning was addressed in [22] that made optimal control decisions for HVAC and window systems to minimize both energy consumption and thermal discomfort. Recently, the latest methods on RL, for example, deep reinforcement learning control [23,24], regularized fitted Q-iteration approach [25], proximal actor critic [26], and auto-encoder [27,28] were appeared in air-conditioning to minimize both energy consumption and thermal discomfort. However, all the studies do not involve the time-varying and uncertain loads.…”

Section: Background and Motivationmentioning

confidence: 99%

Improvement of Refrigeration Efficiency by Combining Reinforcement Learning with a Coarse Model

Zhang

2019

Processes

View full text Add to dashboard Cite

It is paramount to improve operational conversion efficiency in air-conditioning refrigeration. It is noticed that control efficiency for model-based methods highly relies on the accuracy of the mechanism model, and data-driven methods would face challenges using the limited collected data to identify the information beyond. In this study, a hybrid novel approach is presented, which is to integrate a data-driven method with a coarse model. Specifically, reinforcement learning is used to exploit/explore the conversion efficiency of the refrigeration, and a coarse model is utilized to evaluate the reward, by which the requirement of the model accuracy is reduced and the model information is better used. The proposed approach is implemented based on a hierarchical control strategy which is divided into a process level and a loop level. The simulation of a test bed shows the proposed approach can achieve better conversion efficiency of refrigeration than the conventional methods.

show abstract

Practical implementation and evaluation of deep reinforcement learning control for a radiant heating system

Cited by 77 publications

References 22 publications

Evaluating the Adaptability of Reinforcement Learning Based HVAC Control for Residential Houses

Evaluating the Adaptability of Reinforcement Learning Based HVAC Control for Residential Houses

Solar+ Optimizer: A Model Predictive Control Optimization Platform for Grid Responsive Building Microgrids

Improvement of Refrigeration Efficiency by Combining Reinforcement Learning with a Coarse Model

Contact Info

Product

Resources

About