Improving Cross-Subject Activity Recognition via Adversarial Learning

Abstract

Deep learning has been widely used for implementing human activity recognition from wearable sensors like inertial measurement units. The performance of deep activity recognition is heavily affected by the amount and variability of the labeled data available for training the deep learning models. On the other hand, it is costly and time-consuming to collect and label data. Given limited training data, it is hard to maintain high performance across a wide range of subjects, due to the differences in the underlying data distribution of the training and the testing sets. In this work, we develop a novel solution that applies adversarial learning to improve cross-subject performance by generating training data that mimic artificial subjects-i.e. through data augmentation-and enforcing the activity classifier to ignore subject-dependent information. Contrary to domain adaptation methods, our solution does not utilize any data from subjects of the test set (or target domain). Furthermore, our solution is versatile as it can be utilized together with any deep neural network as the classifier. Considering the open dataset PAMAP2, nearly 10% higher crosssubject performance in terms of F1-score can be achieved when training a CNN-LSTM-based classifier with our solution. A performance gain of 5% is also observed when our solution is applied to a stateof-the-art HAR classifier composed of a combination of inception neural network and recurrent neural network. We also investigate different influencing factors of classification performance (i.e. selection of sensor modalities, sampling rates and the number of subjects in the training data), and summarize a practical guideline for implementing deep learning solutions for sensor-based human activity recognition.

Robust myoelectric pattern recognition methods for reducing users’ calibration burden: challenges and future

“…There are also many MPR algorithms based on adversarial learning which strategy is one of the most popular deep learning-based MPR algorithms (Y. Hu et al, 2019 ; Leite and Xiao, 2020 ; Choi et al, 2022 ). Adversarial learning strategies facilitate the unsupervised training of domain-invariant features between domains.…”

Comparing Cross-Subject Performance on Human Activities Recognition Using Learning Models

“…(2) Unsupervised domain adaptation aligns the feature distributions between source and target domains by means of distance minimization [53]- [56] or generative adversarial networks (GAN) [57]- [60]. Hosseini et al [53] designed a BLSTM to extract representative features and minimize confusion between source and target domains through maximum mean discrepancy loss.…”

“…Meanwhile, the adversarial domain generalization method was used in [17], [60] for cross-subject recognition. Only the labeled data of training subjects were used to extract domain invariant features which were independent of subjects through adversarial learning, thus the model had good generalization performance on different but similar domains.…”

“…In Table 10, only [60] used the same strict LOSO criterion with training, validation, and testing as mentioned in Section III-B where the hyper-parameters are transparent to the testing set, while in other LOSO-based studies the model are determined by the best result on the testing set. It worth noting that in the domain adaptation research [51], [52], [54], [55], [58], the labeled and unlabeled target samples were used when training the models, while the other studies had only training data for constructing the classifiers.…”

Fully Connected Generative Adversarial Network for Human Activity Recognition

“…There are two types of sensor data augmentation for activity recognition: basic data augmentation and deep learning data augmentation methods. Basic data augmentation techniques use conventional algorithms to perform different data augmentation techniques that add noise to the sensor readings, increase/decrease the magnitude of the sensor readings, or flip the sign of the original sensor data [17]. One drawback of the basic data augmentation method is generating only limited samples that are not suitable for training deep learning models [18].…”