Real-time Recognition of Interleaved Activities Based on Ensemble Classifier of Long Short-Term Memory with Fuzzy Temporal Windows

Medina-Quero, Javier; Orr, Claire; Zang, Shuai; Nugent, Chris; Salguero, Alberto; Espinilla, Macarena

doi:10.3390/proceedings2191225

Cited by 10 publications

(7 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To generate the input datasets by preprocessing the raw sensor data, multiple and incremental fuzzy temporal windows (FTW) are used. FTW is used as a successful technique to segment the sensor data and prepare the input datasets [4,6,9,18,57]. FTW has shown that it can capture signal sensors of a long and short duration of human activities such as sleep or snack from raw sensor data [4,57].…”

Section: Preprocessing Smart Home Datamentioning

confidence: 99%

Dilated causal convolution with multi-head self attention for sensor human activity recognition

Hamad

Kimura²,

Yang

et al. 2021

Neural Comput & Applic

View full text Add to dashboard Cite

Systems of sensor human activity recognition are becoming increasingly popular in diverse fields such as healthcare and security. Yet, developing such systems poses inherent challenges due to the variations and complexity of human behaviors during the performance of physical activities. Recurrent neural networks, particularly long short-term memory have achieved promising results on numerous sequential learning problems, including sensor human activity recognition. However, parallelization is inhibited in recurrent networks due to sequential operation and computation that lead to slow training, occupying more memory and hard convergence. One-dimensional convolutional neural network processes input temporal sequential batches independently that lead to effectively executed operations in parallel. Despite that, a one-dimensional Convolutional Neural Network is not sensitive to the order of the time steps which is crucial for accurate and robust systems of sensor human activity recognition. To address this problem, we propose a network architecture based on dilated causal convolution and multi-head self-attention mechanisms that entirely dispense recurrent architectures to make efficient computation and maintain the ordering of the time steps. The proposed method is evaluated for human activities using smart home binary sensors data and wearable sensor data. Results of conducted extensive experiments on eight public and benchmark HAR data sets show that the proposed network outperforms the state-of-the-art models based on recurrent settings and temporal models.

show abstract

Section: Preprocessing Smart Home Datamentioning

confidence: 99%

Dilated causal convolution with multi-head self attention for sensor human activity recognition

Hamad

Kimura²,

Yang

et al. 2021

Neural Comput & Applic

View full text Add to dashboard Cite

show abstract

“…To develop intelligent systems from sensor data streams, a new approach was introduced for online AR with three temporal sub-windows [50][51]. Moreover, an ensemble of activity-based classifiers was presented for balanced training and the selection of relevant sensors using FTWs [52]. Even though the fuzzy temporal feature extraction-based approaches in [47][48][49][50][51][52] exhibited advanced performance, performance degradation owing to the unlabeled data for other activities that comprise half of the dataset and present high correlation to the labeled data were not investigated.…”

Section: A Temporal Dependency-based Feature Extractionmentioning

confidence: 99%

“…However, long-term historical data are significantly vast, for example, several hours or a day before query time, and they should be compressed and transformed into a single feature matrix to be effectively learned. In this study, fuzzy features extracted by fuzzy temporal windows (FTWs) [47][48][49][50][51][52], which is a data representation method that effectively compresses large preceding data, were used.…”

Section: Introductionmentioning

confidence: 99%

Deep Learning for Human Activity Recognition Based on Causality Feature Extraction

et al. 2021

View full text Add to dashboard Cite

We propose a novel data-driven feature extraction approach based on direct causality and fuzzy temporal windows (FTWs) to improve the precision of human activity recognition and mitigate the problems of easily-confused activities and unlabeled data, which significantly degrade classification performance owing to the correlation of labeled data. In recognizing activities, the proposed approach not only considers the importance of oncoming short-term sensor data but also considers the continuity from past activities of the preceding long-term sensor data. In terms of the oncoming data, the causality feature is extracted using the direct transfer entropy to determine the unique pattern of an activity, which represents the quantified causal relationship between sensor activations. In terms of the preceding data, several hours of historical data are compressed to fuzzy features based on FTWs. Subsequently, the causality and fuzzy features are fused by matrix multiplication to express distinct features of activities. To effectively learn the spatiotemporal dependencies of the fused feature, deep long short-term memory (LSTM), two-dimensional convolutional neural network (2D-CNN), and hybrid models composed of a combination of LSTM and CNN were used. Leave-one-day-out cross-validation was performed based on the CASAS open datasets, including Aruba, Cairo, and Milan. The results showed that the macro-F1-scores were improved by 16.4, 37.5, and 18.5%, respectively, compared with those of the FTW-only environments. In addition, the proposed approach could improve the precision of activity recognition and mitigate the problems associated with the environments containing unlabeled data.INDEX TERMS activity of daily living, activity recognition, causality, convolutional neural network, deep learning, fuzzy feature, long short-term memory.

show abstract

“…An ensemble of long short-term memory (LSTM) with fuzzy temporal windows method was proposed to solve real-time recognition of interleaved activities in [37]. The authors proposed a representation of binary-sensor activations based on multiple fuzzy temporal windows and then trained an ensemble of LSTM [38] classifiers.…”

Section: Related Workmentioning

confidence: 99%

Radial Basis Function Neural Network with Localized Stochastic-Sensitive Autoencoder for Home-Based Activity Recognition

Wang

et al. 2020

Sensors

Self Cite

View full text Add to dashboard Cite

Over the past few years, the Internet of Things (IoT) has been greatly developed with one instance being smart home devices gradually entering into people's lives. To maximize the impact of such deployments, home-based activity recognition is required to initially recognize behaviors within smart home environments and to use this information to provide better health and social care services. Activity recognition has the ability to recognize people's activities from the information about their interaction with the environment collected by sensors embedded within the home. In this paper, binary data collected by anonymous binary sensors such as pressure sensors, contact sensors, passive infrared sensors etc. are used to recognize activities. A radial basis function neural network (RBFNN) with localized stochastic-sensitive autoencoder (LiSSA) method is proposed for the purposes of home-based activity recognition. An autoencoder (AE) is introduced to extract useful features from the binary sensor data by converting binary inputs into continuous inputs to extract increased levels of hidden information. The generalization capability of the proposed method is enhanced by minimizing both the training error and the stochastic sensitivity measure in an attempt to improve the ability of the classifier to tolerate uncertainties in the sensor data. Four binary home-based activity recognition datasets including OrdonezA, OrdonezB, Ulster, and activities of daily living data from van Kasteren (vanKasterenADL) are used to evaluate the effectiveness of the proposed method. Compared with well-known benchmarking approaches including support vector machine (SVM), multilayer perceptron neural network (MLPNN), random forest and an RBFNN-based method, the proposed method yielded the best performance with 98.35%, 86.26%, 96.31%, 92.31% accuracy on four datasets, respectively. which can be used to underpin a multitude of services such as personal health management [2], elderly care services [3], smart home services [4,5], to name but a few. However, it is a complicated process with many unsolved challenges such as multiple occupancy, interleaved activities, incomplete sensor data and differences in inter-and intra-inhabitant behaviors. As a result, due to the diversity of its data sources (smart home, smart wearable devices, mobile phones sensors and cameras), various solutions have been proposed to solve this problem [6].Some research studies in the area of activity recognition have been based on image data collected by cameras [7][8][9][10]. Limited by computational complexity and privacy issues, this type of activity recognition is mainly used in the field of public safety and human-computer interaction. As an alternative to image-based data, some research has been based on data collected from wearable devices and smartphones [11][12][13]. In this paper, we focus on home-based activity recognition, which is mainly used in elderly care service or smart home service provision. Binary data were collected through binary sensors embedded ...

show abstract

Real-time Recognition of Interleaved Activities Based on Ensemble Classifier of Long Short-Term Memory with Fuzzy Temporal Windows

Cited by 10 publications

References 22 publications

Dilated causal convolution with multi-head self attention for sensor human activity recognition

Dilated causal convolution with multi-head self attention for sensor human activity recognition

Deep Learning for Human Activity Recognition Based on Causality Feature Extraction

Radial Basis Function Neural Network with Localized Stochastic-Sensitive Autoencoder for Home-Based Activity Recognition

Contact Info

Product

Resources

About