Semi-supervised and personalized federated activity recognition based on active learning and label propagation

Abstract

One of the major open problems in sensor-based Human Activity Recognition (HAR) is the scarcity of labeled data. Among the many solutions to address this challenge, semi-supervised learning approaches represent a promising direction. However, their centralized architecture incurs in the scalability and privacy problems that arise when the process involves a large number of users. Federated learning (FL) is a promising paradigm to address these problems. However, the FL methods that have been proposed for HAR assume that the participating users can always obtain labels to train their local models (i.e., they assume a fully supervised setting). In this work, we propose FedAR: a novel hybrid method for HAR that combines semi-supervised and federated learning to take advantage of the strengths of both approaches. FedAR combines active learning and label propagation to semi-automatically annotate the local streams of unlabeled sensor data, and it relies on FL to build a global activity model in a scalable and privacy-aware fashion. FedAR also includes a transfer learning strategy to fine-tune the global model on each user. We evaluated our method on two public datasets, showing that FedAR reaches recognition rates and personalization capabilities similar to state-of-the-art FL supervised approaches. As a major advantage, FedAR only requires a very limited number of annotated data to populate a pre-trained model and a small number of active learning questions that quickly decrease while using the system, leading to an effective and scalable solution for the data scarcity problem of HAR.

FedProp: Cross-client Label Propagation for Federated Semi-supervised Learning

“…FedSem+, which is based on FedSem [Albaseer et al, 2020], computes all pseudo-labels only once during training, but it additionally uses entropy-based sample weights, as we found this to consistently improve performance. FedAvg+PL(localLP), based on FedHar [Presotto et al, 2022], uses per-client label propagation, otherwise it is identical to Algorithm 3. Additionally, we report results for training in a supervised manner on only the available labeled examples, FedAvg (labeled only).…”

“…In this setting Zhang et al [2021] combine local consistency with grouping of client updates to reduce gradient diversity while Diao et al [2022] combine consistency, through strong data augmentation, with pseudo-labeling unlabeled client data. Other methods focus exclusively on pseudo-labeling: Albaseer et al [2020] and Lin et al [2021] both use network predictions to assign pseudo labels, while Presotto et al [2022] develop a specialized method for human activity recognition which uses label propagation locally on each client to pseudo label incoming data. Unlike in classical SSL, none of the above methods fully make use of the knowledge gained from estimating the data manifold because they exploit interactions between data points at most locally within each client.…”

SemiPFL: Personalized Semi-Supervised Federated Learning Framework for Edge Intelligence

“…1. In personalized federated learning, users collaborate to develop personalized models instead of generating a global model [33], [38], [51]- [56]. However, these works primarily focus on image classification benchmarks such as CIFAR10 [57], and CIFAR100 [58], and applications such as multi-sensor and IoT classifications are left less unattended.…”

“…In semisupervised learning, we have a small amount of labeled data and a large amount of unlabeled data. There have been multiple attempts to unify federated learning with semi-supervised learning [12], [33], [38], [9]. Zhao et al consider users have a large amount of unlabeled data, while the server has a set of labeled data [12].…”

“…In federated learning frameworks, users collaboratively train a global model while keeping their data isolated on the edge devices [34]- [36]. The first method introduced by Google in 2016, FedAVG [37], is widely used as the baseline [33], [38], [39]. In FedAVG, the server first sends a global model to each user.…”