SelfHAR

Abstract

Machine learning and deep learning have shown great promise in mobile sensing applications, including Human Activity Recognition. However, the performance of such models in real-world settings largely depends on the availability of large datasets that captures diverse behaviors. Recently, studies in computer vision and natural language processing have shown that leveraging massive amounts of unlabeled data enables performance on par with state-of-the-art supervised models. In this work, we present SelfHAR, a semi-supervised model that effectively learns to leverage unlabeled mobile sensing datasets to complement small labeled datasets. Our approach combines teacher-student self-training, which distills the knowledge of unlabeled and labeled datasets while allowing for data augmentation, and multi-task self-supervision, which learns robust signal-level representations by predicting distorted versions of the input. We evaluated SelfHAR on various HAR datasets and showed state-of-the-art performance over supervised and previous semi-supervised approaches, with up to 12% increase in F1 score using the same number of model parameters at inference. Furthermore, SelfHAR is data-efficient, reaching similar performance using up to 10 times less labeled data compared to supervised approaches. Our work not only achieves state-of-the-art performance in a diverse set of HAR datasets, but also sheds light on how pre-training tasks may affect downstream performance.

Self-Supervised Learning-Based General Laboratory Progress Pretrained Model for Cardiovascular Event Detection

“…This entails transfer learning, a concept that improves a classifier in one domain (pretext task) using more readily obtainable data and then applying this acquired knowledge to another domain (downstream task) [24] . For instance, Tang et al [25] utilized a teacher-student self-training model to capture information from a large-scale unlabeled dataset of wearable and mobile sensing data, which was subsequently transferred to seven different datasets with varying sensor types, populations, and protocols. Similarly, Spathis et al [26] used activity accelerometer sensor data as input to forecast heart rate and transferred the learned representations to capture physiologically meaningful and personalized information using linear classifiers.…”

IoT-assisted Human Activity Recognition Using Bat Optimization Algorithm with Ensemble Voting Classifier for Disabled Persons

“…The HAR problem, concerning machine learning, is considered a classifier task (Brishtel et al, 2023), where the task of the method is to relate all input dataset points to sort out a predefined number of categories (e.g. human actions) (Tang et al, 2021). Classification refers to a supervised task that Journal of Disability Research 2024 the methods learn in the training phase by linking the input dataset to its respective ground truth label that denotes the activity that was happening when the data was recorded.…”

Building generalizable deep learning solutions for mobile sensing

“…Many surrogate tasks [17,18,[50][51][52][53] have been developed to learn spatial and temporal relations from images, videos, and text, proving invaluable in extracting high-level representations that aid downstream transfer and semi-supervised learning models. However, applying these paradigms to mobile sensing poses challenges due to the unique characteristics of sensor data and the limited computational resources of mobile device Recent self-supervised learning initiatives [1,[54][55][56][57][58][59] have demonstrated success in extracting useful features from unlabeled sensor data, thereby enhancing downstream task-specific model performance. For example, the encoder models from TPN [1] and Self-HAR [55] represent early efforts to contextualize features from unlabeled sensor data, enabling task-specific models to achieve improved performance even with limited labeled data.…”

“…However, applying these paradigms to mobile sensing poses challenges due to the unique characteristics of sensor data and the limited computational resources of mobile device Recent self-supervised learning initiatives [1,[54][55][56][57][58][59] have demonstrated success in extracting useful features from unlabeled sensor data, thereby enhancing downstream task-specific model performance. For example, the encoder models from TPN [1] and Self-HAR [55] represent early efforts to contextualize features from unlabeled sensor data, enabling task-specific models to achieve improved performance even with limited labeled data. Nevertheless, these models often still require some labeled data for training downstream classifiers and may risk overfitting to specific domains, limiting their generalizability for target users or environments that lack any labeled data.…”

“…The data augmentation techniques for images or text have been well-established [21][22][23]67] and prior works [68,69] devise a range of data augmentation methods for sensor data, e.g., random noising, to increase label size and prevent seeing model overfitting to specific domains. Recent studies [1,[55][56][57] have been investigating self-supervised learning combined with data augmentation techniques to utilize unlabeled data effectively.…”