Skin Temperature Extraction Using Facial Landmark Detection and Thermal Imaging for Comfort Assessment

Abstract

Despite the large share of energy consumption, current HVAC systems in buildings fail to meet their primary purpose of maintaining comfortable indoor conditions. Current "one size fits all" approach to control the thermal conditions in an environment lead to a high degree of occupant dissatisfaction. Advancements in Internet of Things and Machine Learning have opened the possibility of deploying different sensors at a wide scale to monitor environmental and physiological information and using collected sensor data to model individual comfort requirements. Thermal imaging has recently gained interest as one of the possible ways to monitor physiological information (skin temperature) for thermal comfort assessment. Previous studies have shown that skin temperatures from different regions of the face, such as forehead, nose, cheeks and ears can provide useful information for predicting thermal sensation at an individual level. However, existing approaches to process thermal images either rely on manual temperature extraction or use methods that are less reliable in accurately identifying different facial regions. One of the major challenges of using thermal imaging for monitoring skin temperatures in actual buildings is that occupants may move relative to the camera. It is not practical to expect building occupants to be oriented facing the cameras at all times, therefore, it is important to be able to extract as much information as possible from instances where it is feasible to extract relevant information. In this paper, we describe an approach to extract skin temperature by locating specific regions of the face in thermal images. The approach involves combining data from RGB images with thermal images and leveraging facial landmark detection in RGB images. We also evaluate our approach with existing approach of face detection used in previous studies. Our study demonstrates that facial landmark detection provides a more accurate calculation of different locations in the face compared to previous studies. We show an improvement in overall quantity and quality of temperature measurements extracted from thermal images compared to previous studies. More accurate temperature measurements from thermal images can improve the accuracy of thermal imaging for modeling and predicting thermal comfort.

Environmental conditions, mobile digital culture, mobile usability, knowledge of app in COVID-19 risk mitigation: A structural equation model analysis

“…Congruence to the present study is the syndromic surveillance initiative by the Jonh Hopkins University COVID-cloud control mechanism of smartphone application for collection of body temperature with spatio-temporal clustering techniques ( Güemes et al, 2021 ). Also, the smartphone driven thermometer application for real-time groups population ( Aryal & Becerik-Gerber, 2019a , 2019b ).…”

Thermal Infrared Imaging to Evaluate Emotional Competences in Nursing Students: A First Approach through a Case Study

“…Therefore, for our study, which is closer to a classification problem, the accuracy of temperature measurement using IR camera did not influence significantly the study as it could have done in a regression problem. Regarding the use of low precision camera, authors would like to add that, in Li et al [40] and Aryal and Becerik-Gerber [39], a very low-cost Flir Lepton camera with an accuracy of ±5 • C and 80 × 60 pixels was used for ROI temperature extraction, and authors showed that the camera was adequate for monitoring changes in skin temperature for thermal comfort assessment.…”

Application of Low-Cost Sensors for Building Monitoring: A Systematic Literature Review

“…This technique can save the energy consumption of buildings up to 30%. For the same purpose, FLIR Lepton low-cost thermal camera with accuracy of 2 • C and resolution of 0.1 • C and RGB camera with was used by Aryal et al to extract people skin temperature [146]. Real time monitoring of skin temperature through FLIR thermal camera S65-HS reported by Vissers and Zeiler [147].…”