ColloSSL: Collaborative Self-Supervised Learning for Human Activity Recognition

Abstract

A major bottleneck in training robust Human-Activity Recognition models (HAR) is the need for large-scale labeled sensor datasets. Because labeling large amounts of sensor data is an expensive task, unsupervised and semi-supervised learning techniques have emerged that can learn good features from the data without requiring any labels. In this paper, we extend this line of research and present a novel technique called Collaborative Self-Supervised Learning (ColloSSL) which leverages unlabeled data collected from multiple devices worn by a user to learn high-quality features of the data. A key insight that underpins the design of ColloSSL is that unlabeled sensor datasets simultaneously captured by multiple devices can be viewed as natural transformations of each other, and leveraged to generate a supervisory signal for representation learning. We present three technical innovations to extend conventional self-supervised learning algorithms to a multi-device setting: a Device Selection approach which selects positive and negative devices to enable contrastive learning, a Contrastive Sampling algorithm which samples positive and negative examples in a multi-device setting, and a loss function called Multi-view Contrastive Loss which extends standard contrastive loss to a multi-device setting. Our experimental results on three multi-device datasets show that ColloSSL outperforms both fully-supervised and semi-supervised learning techniques in majority of the experiment settings, resulting in an absolute increase of upto 7.9% in 𝐹 1 score compared to the best performing baselines. We also show that ColloSSL outperforms the fully-supervised methods in a low-data regime, by just using one-tenth of the available labeled data in the best case. CCS Concepts: • Computing methodologies → Unsupervised learning; • Hardware → Sensor applications and deployments; • Human-centered computing → Ubiquitous computing.

FLAME: Federated Learning Across Multi-device Environments

“…In the context of human-activity recognition, an example of MDE is when a person wears multiple inertial sensing devices on their body [23] as shown in Figure 1b. These multiple devices observe the user's activity or context simultaneously and record sensor data in a time-aligned manner [28,70]. In contrast, conventional federated learning setups assume that each user (or client) has a single data-producing device as shown in Figure 1a.…”

“…From a sensing perspective, multi-device environments offer exciting opportunities to develop accurate and generalizable models by leveraging the similarities and differences across devices. Although there have been prior works on training machine learning models for multi-device environments [28,52,70], they were primarily based on centralized training and required sharing of raw data between devices. Applying federated learning to these setting could be a potential privacy-preserving solution, however to the best of our knowledge no prior works have investigated federated learning in these settings.…”