Thermal and Energy Management Based on Bimodal Airflow-Temperature Sensing and Reinforcement Learning

Zhang, Zhen; Ma, Cheng; Zhu, Rong

doi:10.3390/en11102575

Cited by 7 publications

(6 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Cost & Load Balance Mixed/NA [197], [198], [199] Cost & Comfort [200] Other/Mixed Residential A2C [201] HVAC, Fans, WH Cost Commercial A3C [202] P2P Trading Mixed/NA TD3 [203] HVAC, Fans, WH [204] Cost & Comfort [205] Other/Mixed Residential…”

Section: Referencementioning

confidence: 99%

Reinforcement Learning: Theory and Applications in HEMS

Alani¹,

Das²

2022

Preprint

View full text Add to dashboard Cite

The twin capabilities of learning from experience and learning at higher levels of abstraction, set reinforcement learning apart from other areas of machine learning and (within the broader context) all of artificial intelligence. It allows algorithmic agents to replace human beings in the real world, including in homes and buildings, in application domains that had hitherto been considered to be beyond today’s capabilities. This goal, specifically aimed at home energy automation that forms the backdrop of this article, which surveys the use of deep reinforcement learning in various HEMS applications. The article provides an overview of generic reinforcement learning. This is followed with discussions on the state-of-the-art methods for value based, policy gradient, and actor-critic methods in deep reinforcement learning. In order to make published literature in reinforcement learning more accessible to HEMS researchers, verbal descriptions are accompanied with explanatory figures as well as mathematical expressions using the same terminology as the machine learning community. Next, a detailed survey of how reinforcement learning is used in different HEMS domains is described. The survey also considers what kind of reinforcement learning algorithms are used in each HEMS application. The survey suggests that this research is still in its infancy.

show abstract

Section: Referencementioning

confidence: 99%

Reinforcement Learning: Theory and Applications in HEMS

Alani¹,

Das²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Building Type Algorithm [152,153] Other/Mixed Cost Residential DQN [154] Cost and Load Balance [105] EV, ES, and RG Cost [155] Other [156] Cost and Comfort [157] HVAC, Fans, WH Cost [158] Other/Mixed Commercial [159] Cost and Comfort [160,161] EV, ES, and RG [175] Other/Mixed Cost [176] Cost and Comfort Residential TRPO Other/Mixed [182,183] Cost and Load Balance [184] Cost [185] EV, ES, and RG [186] Other/Mixed Cost and Comfort Academic [187] Other [188,189] EV, ES, and RG Commercial [190][191][192] HVAC, Fans, WH Cost and Comfort Mixed/NA [193][194][195] EV, ES, and RG Other [196,197] Other/Mixed Cost and Load Balance Residential SAC [198,199] HVAC, Fans, WH Cost Commercial [103,[200][201][202] Cost and Comfort [203] Other/Mixed [204] Academic [205][206][207] HVAC, Fans, WH Cost and Load Balance Mixed/NA [208][209][210]<...…”

Section: Reference Application Objectivementioning

confidence: 99%

Reinforcement Learning: Theory and Applications in HEMS

Alani¹,

Das²

2022

Preprint

View full text Add to dashboard Cite

The steep rise in reinforcement learning (RL) in various applications in energy as well as the penetration of home automation in recent years are the motivation for this article. It surveys the use of RL in various home energy management system (HEMS) applications. There is a focus on deep neural network (DNN) models in RL. The article provides an overview of reinforcement learning. This is followed with discussions on state-of-the-art methods for value, policy, and actor–critic methods in deep reinforcement learning (DRL). In order to make the published literature in reinforcement learning more accessible to the HEMS community, verbal descriptions are accompanied with explanatory figures as well as mathematical expressions using standard machine learning terminology. Next, a detailed survey of how reinforcement learning is used in different HEMS domains is described. The survey also considers what kind of reinforcement learning algorithms are used in each HEMS application. It suggests that research in this direction is still in its infancy. Lastly, the article proposes four performance metrics to evaluate RL methods.

show abstract

“…A significant amount of published research focuses on temperature regulation by controlling the HVAC system [15][16][17]. However, other published research focuses on controlling other equipment to manage the indoor temperature, such as the air conditioner alone [18], ventilation fan [19,20], underfloor heating [21], heat pumps [22], and the opening and closing of windows [23]. The main objective of such research is to minimize the energy cost while maintaining or increasing occupant [20].…”

Section: Introductionmentioning

confidence: 99%

Imitation Learning with Deep Attentive Tabular Neural Networks for Environmental Prediction and Control in Smart Home

Al-Ani

Das

2023

Energies

View full text Add to dashboard Cite

Automated indoor environmental control is a research topic that is beginning to receive much attention in smart home automation. All machine learning models proposed to date for this purpose have relied on reinforcement learning using simple metrics of comfort as reward signals. Unfortunately, such indicators do not take into account individual preferences and other elements of human perception. This research explores an alternative (albeit closely related) paradigm called imitation learning. In the proposed architecture, machine learning models are trained with tabular data pertaining to environmental control activities of the real occupants of a residential unit. This eliminates the need for metrics that explicitly quantify human perception of comfort. Moreover, this article introduces the recently proposed deep attentive tabular neural network (TabNet) into smart home research by incorporating TabNet-based components within its overall framework. TabNet has consistently outperformed all other popular machine learning models in a variety of other application domains, including gradient boosting, which was previously considered ideal for learning from tabular data. The results obtained herein strongly suggest that TabNet is the best choice for smart home applications. Simulations conducted using the proposed architecture demonstrate its effectiveness in reproducing the activity patterns of the home unit’s actual occupants.

show abstract

Thermal and Energy Management Based on Bimodal Airflow-Temperature Sensing and Reinforcement Learning

Cited by 7 publications

References 28 publications

Reinforcement Learning: Theory and Applications in HEMS

Reinforcement Learning: Theory and Applications in HEMS

Reinforcement Learning: Theory and Applications in HEMS

Imitation Learning with Deep Attentive Tabular Neural Networks for Environmental Prediction and Control in Smart Home

Contact Info

Product

Resources

About