Activity Recognition in Residential Spaces with Internet of Things Devices and Thermal Imaging

Naik, Kshirasagar; Pandit, Tejas; Naik, Nitin; Shah, Parth

doi:10.3390/s21030988

Cited by 18 publications

(13 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Camera-based (infrared and optical cameras) are used in [3,8,9,34,35,36] alongside machine learning to carefully analyze capture image frames for occupancy detection and estimation in commercial and residential buildings. The fusion modalities are considered to differentiate human occupancy and other object emitting thermal heat in the environment and support night vision prediction.…”

Section: Literature Reviewmentioning

confidence: 99%

Comparative Analysis of Machine Learning Methods for Non-Intrusive Indoor Occupancy Detection and Estimation

Aliero¹,

Pasha²,

Toosi³

et al. 2021

Preprint

View full text Add to dashboard Cite

Occupancy-driven application research has been active research for a decade that focuses on improving or replacing new building infrastructure to improve building energy efficiency. Existing approaches for HVAC energy saving are putting more emphasis on occupancy detection, estimation, and localization to trade-off between energy consumption and thermal comfort satisfaction. In a non-intrusive approach, various sensors, actuators, and analytic data methods are commonly used to process data from occupant surroundings and trigger appropriate action to achieve the task. However, the performance of the non-intrusive approach reported in the literature is relatively poor due to the lack of quality of dataset used in model training and expropriate choice of machine learning model. This study proposed a non-intrusive approach that to improve the collection and quality of dataset using data pre-processing. The study collected a training dataset using various sensors installed in the building and developed a model using five machine learning models to determine occupant’s presence and estimate their number in the building. The proposed solution is tested in the living room with a prototype system integrated with various sensors designed to obtain occupant surrounding environmental datasets. The model’s prediction results obtained indicate that it is possible for the proposed solution to obtain data, process, and predict the occupant number with high accuracy (73.6 -99.7% using random forest).

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Comparative Analysis of Machine Learning Methods for Non-Intrusive Indoor Occupancy Detection and Estimation

Aliero¹,

Pasha²,

Toosi³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…However, human activity recognition using only CCD RGB-IR with the restricted texture details will not have perfect recognition performances. In some studies [ 20 , 21 , 22 , 23 ], thermal image-based approaches to use images of infrared thermography (IRT) acquired from the sensing device of the IR thermal imager for analyzing activity gesture information are presented to maintain a satisfactory recognition accuracy even in an adverse environment of low lights. However, a high cost problem of the IR thermal imager will be inevitably encountered, which will be an adverse factor on the market acceptance to such intelligent applications.…”

Section: Introductionmentioning

confidence: 99%

CNN Deep Learning with Wavelet Image Fusion of CCD RGB-IR and Depth-Grayscale Sensor Data for Hand Gesture Intention Recognition

Ding

Zheng

2022

Sensors

View full text Add to dashboard Cite

Pixel-based images captured by a charge-coupled device (CCD) with infrared (IR) LEDs around the image sensor are the well-known CCD Red–Green–Blue IR (the so-called CCD RGB-IR) data. The CCD RGB-IR data are generally acquired for video surveillance applications. Currently, CCD RGB-IR information has been further used to perform human gesture recognition on surveillance. Gesture recognition, including hand gesture intention recognition, is attracting great attention in the field of deep neural network (DNN) calculations. For further enhancing conventional CCD RGB-IR gesture recognition by DNN, this work proposes a deep learning framework for gesture recognition where a convolution neural network (CNN) incorporated with wavelet image fusion of CCD RGB-IR and additional depth-based depth-grayscale images (captured from depth sensors of the famous Microsoft Kinect device) is constructed for gesture intention recognition. In the proposed CNN with wavelet image fusion, a five-level discrete wavelet transformation (DWT) with three different wavelet decomposition merge strategies, namely, max-min, min-max and mean-mean, is employed; the visual geometry group (VGG)-16 CNN is used for deep learning and recognition of the wavelet fused gesture images. Experiments on the classifications of ten hand gesture intention actions (specified in a scenario of laboratory interactions) show that by additionally incorporating depth-grayscale data into CCD RGB-IR gesture recognition one will be able to further increase the averaged recognition accuracy to 83.88% for the VGG-16 CNN with min-max wavelet image fusion of the CCD RGB-IR and depth-grayscale data, which is obviously superior to the 75.33% of VGG-16 CNN with only CCD RGB-IR.

show abstract

“…Moreover, commercial solutions cannot detect user interaction with the cooking process. Thus, the researcher community has made significant efforts to develop a more dedicated cooking-monitoring process [ 17 , 18 , 19 , 20 , 24 , 25 , 26 , 27 , 28 ].…”

Section: Introductionmentioning

confidence: 99%

“…One of the possible approaches for monitoring the process in the kitchen is the use of thermal cameras [ 18 , 26 ]. In this way, the temperature of objects in the kitchen can be monitored, primarily the stove.…”

Section: Introductionmentioning

confidence: 99%

“…Another very important aspect of the AAL system is related to the appropriate analysis of the data received from sensors through the decision-making process. Specifically, pure sensor data are sufficient only for triggering alarms for critical situations such as fire detection [ 11 , 14 , 24 , 25 , 26 , 27 , 28 , 29 , 30 ], smoke [ 11 , 14 , 24 , 26 , 27 , 28 , 29 ], and gas leakage [ 11 , 24 , 25 , 29 ]. To obtain information about daily user activity, more complex analysis and decision-making systems are required.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Design of a Kitchen-Monitoring and Decision-Making System to Support AAL Applications

Žarić

Radonjić

Pavlićević

et al. 2021

Sensors

View full text Add to dashboard Cite

Numerous researchers are working on Ambient Assisted Living systems to enable more comfortable and safer living for senior people in their homes. Due to modern lifestyles and an aging population, this has become a very important issue. According to the available literature, it is obvious that the kitchen is one of the most important and most dangerous rooms in the home. However, there is still evident lack of monitoring systems suitable for specific kitchen activities. In this paper, we propose a monitoring system capable of identifying activities related to the cooking process, and a decision-making system capable of identifying some unwanted and possibly critical conditions. The proposed systems are designed to satisfy the requirements of the modern Ambient Assisted Living systems dedicated to older adults. The proposed monitoring system consists of ultrasound, temperature, and humidity sensors. The acquired results from these sensors are the inputs for the decision-making system, which generate warnings or alarms intended for the senior users and/or formal or informal caregivers. This system is designed to improve home safety related to kitchen activities, as well as to provide information about the lifestyle and daily activities of senior users. Experimental validation of the proposed system confirms its functionality and accurate design approach.

show abstract

Activity Recognition in Residential Spaces with Internet of Things Devices and Thermal Imaging

Cited by 18 publications

References 54 publications

Comparative Analysis of Machine Learning Methods for Non-Intrusive Indoor Occupancy Detection and Estimation

Comparative Analysis of Machine Learning Methods for Non-Intrusive Indoor Occupancy Detection and Estimation

CNN Deep Learning with Wavelet Image Fusion of CCD RGB-IR and Depth-Grayscale Sensor Data for Hand Gesture Intention Recognition

Design of a Kitchen-Monitoring and Decision-Making System to Support AAL Applications

Contact Info

Product

Resources

About