Deep convolutional feature transfer across mobile activity recognition domains, sensor modalities and locations

Morales, Francisco; Roggen, Daniel

doi:10.1145/2971763.2971764

Cited by 106 publications

(68 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…activity recognition (HAR), 1D convolutional and recurrent neural networks trained on raw labeled signals signi cantly improve the detection rate over traditional methods [20,44,68,72,73]. Despite the recent advances in the eld of HAR, learning representations from a massive amount of unlabeled data still presents a signi cant challenge.…”

Section: :2 • a Saeed Et Almentioning

confidence: 99%

“…We also demonstrate that the learned representations from a di erent (but related) unlabeled data source can be successfully transferred to improve the performance of diverse tasks even in the case of semi-supervised learning. In terms of transfer learning, our approach also di ers signi cantly from some earlier a empts [44,69] that were concerned with features transferability from a fully-supervised model learned from inertial measurement units data, as our approach utilizes widely available smartphones and does not require labeled data. Finally, the proposed technique is also di erent from previously studied unsupervised pre-training methods such as autoencoders [37], restricted Boltzmann machines [55] and sparse coding [9] as we employ an end-to-end (self) supervised learning paradigm on multiple surrogate tasks to extract features.…”

Section: Determining Representational Similaritymentioning

confidence: 99%

See 1 more Smart Citation

Multi-task Self-Supervised Learning for Human Activity Detection

Saeed

Özçelebi

Lukkien

2019

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

224

199

View full text Add to dashboard Cite

Deep learning methods are successfully used in applications pertaining to ubiquitous computing, pervasive intelligence, health, and well-being. Speci cally, the area of human activity recognition (HAR) is primarily transformed by the convolutional and recurrent neural networks, thanks to their ability to learn semantic representations directly from raw input. However, in order to extract generalizable features massive amounts of well-curated data are required, which is a notoriously challenging task; hindered by privacy issues and annotation costs. erefore, unsupervised representation learning (i.e., learning without manually labeling the instances) is of prime importance to leverage the vast amount of unlabeled data produced by smart devices. In this work, we propose a novel self-supervised technique for feature learning from sensory data that does not require access to any form of semantic labels, i.e., activity classes. We learn a multi-task temporal convolutional network to recognize transformations applied on an input signal. By exploiting these transformations, we demonstrate that simple auxiliary tasks of the binary classi cation result in a strong supervisory signal for extracting useful features for the down-stream task. We extensively evaluate the proposed approach on several publicly available datasets for smartphone-based HAR in unsupervised, semi-supervised and transfer learning se ings. Our method achieves performance levels superior to or comparable with fully-supervised networks trained directly with activity labels, and it performs signi cantly be er than unsupervised learning through autoencoders. Notably, for the semi-supervised case, the self-supervised features substantially boost the detection rate by a aining a kappa score between 0.7 − 0.8 with only 10 labeled examples per class. We get similar impressive performance even if the features are transferred from a di erent data source. Self-supervision drastically reduces the requirement of labeled activity data, e ectively narrowing the gap between supervised and unsupervised techniques for learning meaningful representations. While this paper focuses on HAR as the application domain, the proposed approach is general and could be applied to a wide variety of problems in other areas. 000:2 • A. Saeed et al.activity recognition (HAR), 1D convolutional and recurrent neural networks trained on raw labeled signals signi cantly improve the detection rate over traditional methods [20,44,68,72,73]. Despite the recent advances in the eld of HAR, learning representations from a massive amount of unlabeled data still presents a signi cant challenge. Obtaining large, well-curated activity recognition datasets is problematic due to a number of issues. First, smartphone data are privacy sensitive, which makes it hard to collect su cient amounts of user-activity instances in a real-life se ing. Second, the annotation cost and the time it takes to generate a large volume of labeled instances are prohibitive. Finally, the diversity of devices, types of embedd...

show abstract

Section: :2 • a Saeed Et Almentioning

confidence: 99%

Section: Determining Representational Similaritymentioning

confidence: 99%

Multi-task Self-Supervised Learning for Human Activity Detection

Saeed

Özçelebi

Lukkien

2019

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

224

199

View full text Add to dashboard Cite

show abstract

“…erefore, the features are likely to be more domain-invariant and tend to perform be er for cross-domain tasks. A recent work evaluated deep models for cross-domain HAR [28], which provides some experience for future research on this area. ere are still many open problems for deep learning based CDAR.…”

Section: Related Work 21 Activity Recognitionmentioning

confidence: 99%

Deep Transfer Learning for Cross-domain Activity Recognition

Wang

Zheng

Chen

et al. 2018

Proceedings of the 3rd International Conference on Crowd Science and Engineering

115

View full text Add to dashboard Cite

Human activity recognition plays an important role in people's daily life. However, it is o en expensive and time-consuming to acquire su cient labeled activity data. To solve this problem, transfer learning leverages the labeled samples from the source domain to annotate the target domain which has few or none labels. Unfortunately, when there are several source domains available, it is di cult to select the right source domains for transfer. e right source domain means that it has the most similar properties with the target domain, thus their similarity is higher, which can facilitate transfer learning. Choosing the right source domain helps the algorithm perform well and prevents the negative transfer. In this paper, we propose an e ective Unsupervised Source Selection algorithm for Activity Recognition (USSAR). USSAR is able to select the most similar K source domains from a list of available domains. After this, we propose an e ective Transfer Neural Network to perform knowledge transfer for Activity Recognition (TNNAR). TNNAR could capture both the time and spatial relationship between activities while transferring knowledge. Experiments on three public activity recognition datasets demonstrate that: 1) e USSAR algorithm is e ective in selecting the best source domains. 2) e TNNAR method can reach high accuracy when performing activity knowledge transfer.

show abstract

“…Therefore, the features are likely to be more domain-invariant and tend to perform better for cross-domain tasks. A recent work evaluated deep models for cross-domain HAR [23], which provides some experience for future research on this area. There are stll many open problems for deep learning based CDAR.…”

Section: A Activity Recognitionmentioning

confidence: 99%

Stratified Transfer Learning for Cross-domain Activity Recognition

Wang

Chen

et al. 2018

2018 IEEE International Conference on Pervasive Computing and Communications (PerCom)

183

106

View full text Add to dashboard Cite

In activity recognition, it is often expensive and time-consuming to acquire sufficient activity labels. To solve this problem, transfer learning leverages the labeled samples from the source domain to annotate the target domain which has few or none labels. Existing approaches typically consider learning a global domain shift while ignoring the intra-affinity between classes, which will hinder the performance of the algorithms. In this paper, we propose a novel and general cross-domain learning framework that can exploit the intra-affinity of classes to perform intra-class knowledge transfer. The proposed framework, referred to as Stratified Transfer Learning (STL), can dramatically improve the classification accuracy for cross-domain activity recognition. Specifically, STL first obtains pseudo labels for the target domain via majority voting technique. Then, it performs intra-class knowledge transfer iteratively to transform both domains into the same subspaces. Finally, the labels of target domain are obtained via the second annotation. To evaluate the performance of STL, we conduct comprehensive experiments on three large public activity recognition datasets (i.e. OPPORTU-NITY, PAMAP2, and UCI DSADS), which demonstrates that STL significantly outperforms other state-of-the-art methods w.r.t. classification accuracy (improvement of 7.68%). Furthermore, we extensively investigate the performance of STL across different degrees of similarities and activity levels between domains. And we also discuss the potential of STL in other pervasive computing applications to provide empirical experience for future research.

show abstract

Deep convolutional feature transfer across mobile activity recognition domains, sensor modalities and locations

Cited by 106 publications

References 10 publications

Multi-task Self-Supervised Learning for Human Activity Detection

Multi-task Self-Supervised Learning for Human Activity Detection

Deep Transfer Learning for Cross-domain Activity Recognition

Stratified Transfer Learning for Cross-domain Activity Recognition

Contact Info

Product

Resources

About