Human activity recognition (HAR) is fundamental to many services in smart buildings. However, providing sufficiently robust activity recognition systems that could be confidently deployed in an ordinary real environment remains a major challenge. Much of the research done in this area has mainly focused on recognition through pre-segmented sensor data. In this paper, real-time human activity recognition based on streaming sensors is investigated. The proposed methodology incorporates dynamic event windowing based on spatio-temporal correlation and the knowledge of activity trigger sensor to recognize activities and record new events. The objective is to determine whether the last event that just happened belongs to the current activity, or if it is the sign of the start of a new activity. For this, we consider the correlation between sensors in view of what can be seen in the history of past events. The proposed algorithm contains three steps: verification of sensor correlation (SC), verification of temporal correlation (TC), and determination of the activity triggering the sensor. The proposed approach is applied to a real case study: the “Aruba” dataset from the CASAS database. F1 score is used to assess the quality of the segmentation. The results show that the proposed approach segments several activities (sleeping, bed to toilet, meal preparation, eating, housekeeping, working, entering home, and leaving home) with an F1 score of 0.63–0.99.
Recently, deep learning (DL) approaches have been extensively employed to recognize human activities in smart buildings, which greatly broaden the scope of applications in this field. Convolutional neural networks (CNN), well known for feature extraction and activity classification, have been applied for estimating human activities. However, most CNN-based techniques usually focus on divided sequences associated to activities, since many real-world employments require information about human activities in real time. In this work, an online human activity recognition (HAR) framework on streaming sensor is proposed. The methodology incorporates real-time dynamic segmentation, stigmergy-based encoding, and classification with a CNN2D. Dynamic segmentation decides if two succeeding events belong to the same activity segment or not. Then, because a CNN2D requires a multi-dimensional format in input, stigmergic track encoding is adopted to build encoded features in a multi-dimensional format. It adopts the directed weighted network (DWN) that takes into account the human spatio-temporal tracks with a requirement of overlapping activities. It represents a matrix that describes an activity segment. Once the DWN for each activity segment is determined, a CNN2D with a DWN in input is adopted to classify activities. The proposed approach is applied to a real case study: the “Aruba” dataset from the CASAS database.
In recent years, real-time human activity recognition (HAR) has reached importance due to its applications in various domains such as assistive services for the elderly in smart buildings, monitoring, well-being, comfort and security. Various techniques, researched within the image processing and computer vision communities, have been established to recognize human activities in real-time, but all of them are based on wearable sensors and there is no much attention for ambient sensor based approaches. In the literature, deep learning (DL) is one of effective and cost-efficient supervised learning model and different architectures haves been investigated for real-time HAR. However, it still struggles with the quality of data as well as hardware implementation issues.This paper presents two contributions. Firstly, an intensive analysis of DL architectures and its characteristics along with their limitations in the framework of real time HAR are investigated. Secondly, existing hardware architectures and related challenges in this field are highlighted (adaptation of DL architectures towards microcontrollers, difficulty to provide a smart home with numerous sensors and trends regarding cloud-bases approaches). Then, new research directions and solutions around the real-time data quality assessment, the study of main performance factors for DL on microcontrollers, the concept of minimal sensors set up for the employment of IoT devices and the distributed intelligence are suggested to solve them respectively and to improve this field.
Building Energy Management (BEM) and monitoring systems should not only consider HVAC systems and building physics but also human behaviors. These systems could provide information and advice to occupants about the significance of their practices with regard to the current state of a dwelling. It is also possible to provide services such as assistance to the elderly, comfort and health monitoring. For this, an intelligent building must know the daily activities of its residents and the algorithms of the smart environment must track and recognize the activities that the occupants normally perform as part of their daily routine. In the literature, deep learning is one of effective supervised learning model and cost-efficient for real-time HAR, but it still struggles with the quality of training data (missing values in time series and non-annotated event), the variability of data, the data segmentation and the ontology of activities. In this work, recent research works, existing algorithms and related challenges in this field are firstly highlighted. Then, new research directions and solutions (performing fault detection and diagnosis for drift detection, multi-label classification modeling for multi-occupant classification, new indicators for training data quality, new metrics weighted by the number of representations in dataset to handle the issue of missing data and finally language processing for complex activity recognition) are suggested to solve them respectively and to improve this field.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.