Dynamic Segmentation of Sensor Events for Real-Time Human Activity Recognition in a Smart Home Context

Abstract

Human activity recognition (HAR) is fundamental to many services in smart buildings. However, providing sufficiently robust activity recognition systems that could be confidently deployed in an ordinary real environment remains a major challenge. Much of the research done in this area has mainly focused on recognition through pre-segmented sensor data. In this paper, real-time human activity recognition based on streaming sensors is investigated. The proposed methodology incorporates dynamic event windowing based on spatio-temporal correlation and the knowledge of activity trigger sensor to recognize activities and record new events. The objective is to determine whether the last event that just happened belongs to the current activity, or if it is the sign of the start of a new activity. For this, we consider the correlation between sensors in view of what can be seen in the history of past events. The proposed algorithm contains three steps: verification of sensor correlation (SC), verification of temporal correlation (TC), and determination of the activity triggering the sensor. The proposed approach is applied to a real case study: the “Aruba” dataset from the CASAS database. F1 score is used to assess the quality of the segmentation. The results show that the proposed approach segments several activities (sleeping, bed to toilet, meal preparation, eating, housekeeping, working, entering home, and leaving home) with an F1 score of 0.63–0.99.

Convolutional Neural Network Bootstrapped by Dynamic Segmentation and Stigmergy-Based Encoding for Real-Time Human Activity Recognition in Smart Homes

“…Recognizing human activities is important for many services in smart building. This paper proposed a framework of real-time HAR based on three steps: (1) real-time segmentation of sensor events using the method presented in [ 35 ]; (2) encoding segments into a multidimensional format using the concept of DWN under constraints of non overlapping activities; and (3) classification of the activities using a CNN2D that takes into inputs the DWN of each related activity segment.…”

“…Five methods of fixed-size windowing with different weighting factors are proposed. The work presented in [ 35 ] proposed a method of dynamic segmentation on streaming data that integrated time correlation and event correlation and performed real-time HAR on streaming data.…”

“…In this paper, we used a real-time dynamic segmentation based on the calculation of event spatial correlation [ 35 ] with some rearrangement in the computations. For the case study, an apartment with labeled activities is employed to evaluate our framework.…”

“…This ensures that events from very different zones will not be in the same window. This method is published in [ 35 ] and it allows for determining the beginning and the end of each segment when new events are inscribed.…”

“…The first step of the considered methodology is the real-time dynamic segmentation method, which is investigated in [ 35 ] and which adopts a dynamic window size. Figure 10 shows an example of real-time dynamic segmentation on 4th November 2010.…”