Unsupervised Feature Learning for Human Activity Recognition Using Smartphone Sensors

Li, Yongmou; Shi, Dianxi; Ding, Bo; Liu, Dongbo

doi:10.1007/978-3-319-13817-6_11

Cited by 66 publications

(37 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sensor Modality Deep Model Application Dataset (Almaslukh et al, 2017) Body-worn SAE ADL D03 (Alsheikh et al, 2016) Body-worn RBM ADL, factory, Parkinson D02, D06, D14 Body-worn, ambiemt RBM Gesture, ADL, transportation Self, D01 (Chen and Xue, 2015) Body-worn CNN ADL Self (Chen et al, 2016b) Body-worn CNN ADL D06 (Cheng and Scotland, 2017) Body-worn DNN Parkinson Self (Edel and Köppe, 2016) Body-worn RNN ADL D01, D04, Self (Fang and Hu, 2014) Object, ambient DBN ADL Self (Gjoreski et al, 2016) Body-worn CNN ADL Self, D01 (Guan and Ploetz, 2017) Body-worn, object, ambient RNN ADL, smart home D01, D02, D04 (Ha et al, 2015) Body-worn CNN Factory, health D02, D13 (Ha and Choi, 2016) Body-worn CNN ADL, health D13 (Hammerla et al, 2015) Body-worn RBM Parkinson Self (Hammerla et al, 2016) Body-worn, object, ambient DNN, CNN, RNN ADL, smart home, gait D01, D04, D14 (Hannink et al, 2017) Body-worn CNN Gait Self (Hayashi et al, 2015) Body-worn, ambient RBM ADL, smart home D16 (Inoue et al, 2016) Body-worn RNN ADL D16 (Jiang and Yin, 2015) Body-worn CNN ADL D03, D05, D11 (Khan et al, 2017) Ambient CNN Respiration Self (Kim and Toomajian, 2016) Ambient CNN Hand gesture Self (Kim and Li, 2017) Body-worn CNN ADL Self Body-worn, ambient RBM ADL, emotion Self Ambient RBM ADL Self (Lee et al, 2017) Body-worn CNN ADL Self (Li et al, 2016a) Object RBM Patient resuscitation Self (Li et al, 2016b) Object CNN Patient resuscitation Self (Li et al, 2014) Body-worn SAE ADL D03 Body-worn CNN, RBM ADL Self (Mohammed and Tashev, 2017) Body-worn CNN ADL, gesture Self (Morales and Roggen, 2016) Body-worn CNN ADL, smart home D01, D02 (Murad and Pyun, 2017) Body-worn RNN ADL, smart home D01, D02, D05, D14 (Ordóñez and Roggen, 2016) Body-worn CNN, RNN ADL, gesture, posture, factory D01, D02 (Panwar et al, 2017) Body-worn CNN ADL Self (Plötz et al, 2011) Body-worn, object RBM ADL, food preparation, factory D01, D02, D08, D14…”

Section: Literaturementioning

confidence: 99%

Deep learning for sensor-based activity recognition: A survey

Wang

Chen

Hao

et al. 2019

Pattern Recognition Letters

1,510

927

View full text Add to dashboard Cite

Research Highlights (Required)To create your highlights, please type the highlights against each \item command.It should be short collection of bullet points that convey the core findings of the article. It should include 3 to 5 bullet points (maximum 85 characters, including spaces, per bullet point.)• We survey deep learning based HAR in sensor modality, deep model, and application.• We comprehensively discuss the insights of deep learning models for HAR tasks.• We extensively investigate why deep learning can improve the performance of HAR.• We also summarize the public HAR datasets frequently used for research purpose.• We present some grand challenges and feasible solutions for deep learning based HAR. ABSTRACTSensor-based activity recognition seeks the profound high-level knowledge about human activities from multitudes of low-level sensor readings. Conventional pattern recognition approaches have made tremendous progress in the past years. However, those methods often heavily rely on heuristic hand-crafted feature extraction, which could hinder their generalization performance. Additionally, existing methods are undermined for unsupervised and incremental learning tasks. Recently, the recent advancement of deep learning makes it possible to perform automatic high-level feature extraction thus achieves promising performance in many areas. Since then, deep learning based methods have been widely adopted for the sensor-based activity recognition tasks. This paper surveys the recent advance of deep learning based sensor-based activity recognition. We summarize existing literature from three aspects: sensor modality, deep model, and application. We also present detailed insights on existing work and propose grand challenges for future research.

show abstract

Section: Literaturementioning

confidence: 99%

Deep learning for sensor-based activity recognition: A survey

Wang

Chen

Hao

et al. 2019

Pattern Recognition Letters

1,510

927

View full text Add to dashboard Cite

show abstract

“…In terms of transfer learning, our approach also di ers signi cantly from some earlier a empts [44,69] that were concerned with features transferability from a fully-supervised model learned from inertial measurement units data, as our approach utilizes widely available smartphones and does not require labeled data. Finally, the proposed technique is also di erent from previously studied unsupervised pre-training methods such as autoencoders [37], restricted Boltzmann machines [55] and sparse coding [9] as we employ an end-to-end (self) supervised learning paradigm on multiple surrogate tasks to extract features.…”

Section: Determining Representational Similaritymentioning

confidence: 99%

Multi-task Self-Supervised Learning for Human Activity Detection

Saeed

Özçelebi

Lukkien

2019

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

230

199

View full text Add to dashboard Cite

Deep learning methods are successfully used in applications pertaining to ubiquitous computing, pervasive intelligence, health, and well-being. Speci cally, the area of human activity recognition (HAR) is primarily transformed by the convolutional and recurrent neural networks, thanks to their ability to learn semantic representations directly from raw input. However, in order to extract generalizable features massive amounts of well-curated data are required, which is a notoriously challenging task; hindered by privacy issues and annotation costs. erefore, unsupervised representation learning (i.e., learning without manually labeling the instances) is of prime importance to leverage the vast amount of unlabeled data produced by smart devices. In this work, we propose a novel self-supervised technique for feature learning from sensory data that does not require access to any form of semantic labels, i.e., activity classes. We learn a multi-task temporal convolutional network to recognize transformations applied on an input signal. By exploiting these transformations, we demonstrate that simple auxiliary tasks of the binary classi cation result in a strong supervisory signal for extracting useful features for the down-stream task. We extensively evaluate the proposed approach on several publicly available datasets for smartphone-based HAR in unsupervised, semi-supervised and transfer learning se ings. Our method achieves performance levels superior to or comparable with fully-supervised networks trained directly with activity labels, and it performs signi cantly be er than unsupervised learning through autoencoders. Notably, for the semi-supervised case, the self-supervised features substantially boost the detection rate by a aining a kappa score between 0.7 − 0.8 with only 10 labeled examples per class. We get similar impressive performance even if the features are transferred from a di erent data source. Self-supervision drastically reduces the requirement of labeled activity data, e ectively narrowing the gap between supervised and unsupervised techniques for learning meaningful representations. While this paper focuses on HAR as the application domain, the proposed approach is general and could be applied to a wide variety of problems in other areas. 000:2 • A. Saeed et al.activity recognition (HAR), 1D convolutional and recurrent neural networks trained on raw labeled signals signi cantly improve the detection rate over traditional methods [20,44,68,72,73]. Despite the recent advances in the eld of HAR, learning representations from a massive amount of unlabeled data still presents a signi cant challenge. Obtaining large, well-curated activity recognition datasets is problematic due to a number of issues. First, smartphone data are privacy sensitive, which makes it hard to collect su cient amounts of user-activity instances in a real-life se ing. Second, the annotation cost and the time it takes to generate a large volume of labeled instances are prohibitive. Finally, the diversity of devices, types of embedd...

show abstract

“…Different positions of the mobile phone will cause the direction of the axis to change. In [32] and [33], the authors both eliminate the possible rotational interference by synthesizing the acceleration without considering the direction of the axis, which partly takes into account the relations between the three axes. By converting the time-series signal of the sensor into an active image containing the hidden relations between axes, the recognition accuracy of the model is obviously improved [14].…”

Section: Related Workmentioning

confidence: 99%

The Effect of Axis-Wise Triaxial Acceleration Data Fusion in CNN-Based Human Activity Recognition

Han

Luo

et al. 2020

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

The triaxial accelerometer is one of the most important sensors for human activity recognition (HAR). It has been observed that the relations between the axes of a triaxial accelerometer plays a significant role in improving the accuracy of activity recognition. However, the existing research rarely focuses on these relations, but rather on the fusion of multiple sensors. In this paper, we propose a data fusion-based convolutional neural network (CNN) approach to effectively use the relations between the axes. We design a single-channel data fusion method and multichannel data fusion method in consideration of the diversified formats of sensor data. After obtaining the fused data, a CNN is used to extract the features and perform classification. The experiments show that the proposed approach has an advantage over the CNN in accuracy. Moreover, the single-channel model achieves an accuracy of 98.83% with the WISDM dataset, which is higher than that of state-of-the-art methods.

show abstract

Unsupervised Feature Learning for Human Activity Recognition Using Smartphone Sensors

Cited by 66 publications

References 5 publications

Deep learning for sensor-based activity recognition: A survey

Deep learning for sensor-based activity recognition: A survey

Multi-task Self-Supervised Learning for Human Activity Detection

The Effect of Axis-Wise Triaxial Acceleration Data Fusion in CNN-Based Human Activity Recognition

Contact Info

Product

Resources

About