Negative Selection by Clustering for Contrastive Learning in Human Activity Recognition

Wang, Jinqiang; Zhu, Tao; Chen, Liming; Ning, Huansheng; Wan, Yaping

doi:10.1109/jiot.2023.3239945

Cited by 16 publications

(5 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another study introduced CSSHAR [19], which replaced SimCLR's backbone network with a custom transformer encoder to enhance feature representations extracted from unlabeled sensory data in HAR. ClusterCLHAR [21], following the SimCLR framework, proposed a novel contrastive learning approach for HAR by incorporating negative selection through clustering. The experimental results of ClusterCLHAR show competitive performance in both self-supervised and semi-supervised learning for HAR.…”

Section: Simclr For Human Activity Recognitionmentioning

confidence: 99%

“…The SimCLR framework [18] has proven to be a powerful transfer-based encoder for learning feature representations from unlabeled sensor data in HAR [19]. Several researchers have proposed novel models, including SimCLRHAR [20], CSSHAR [19], and ClusterCLHAR [21], which are based on the SimCLR framework. These models have demonstrated impressive accuracy in HAR tasks using sensor data from smartphones and smartwatches.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Enhancing Human Activity Recognition in Smart Homes with Self-Supervised Learning and Self-Attention

Chen,

Gouin-Vallerand,

Bouchard

et al. 2024

Sensors

View full text Add to dashboard Cite

Deep learning models have gained prominence in human activity recognition using ambient sensors, particularly for telemonitoring older adults’ daily activities in real-world scenarios. However, collecting large volumes of annotated sensor data presents a formidable challenge, given the time-consuming and costly nature of traditional manual annotation methods, especially for extensive projects. In response to this challenge, we propose a novel AttCLHAR model rooted in the self-supervised learning framework SimCLR and augmented with a self-attention mechanism. This model is designed for human activity recognition utilizing ambient sensor data, tailored explicitly for scenarios with limited or no annotations. AttCLHAR encompasses unsupervised pre-training and fine-tuning phases, sharing a common encoder module with two convolutional layers and a long short-term memory (LSTM) layer. The output is further connected to a self-attention layer, allowing the model to selectively focus on different input sequence segments. The incorporation of sharpness-aware minimization (SAM) aims to enhance model generalization by penalizing loss sharpness. The pre-training phase focuses on learning representative features from abundant unlabeled data, capturing both spatial and temporal dependencies in the sensor data. It facilitates the extraction of informative features for subsequent fine-tuning tasks. We extensively evaluated the AttCLHAR model using three CASAS smart home datasets (Aruba-1, Aruba-2, and Milan). We compared its performance against the SimCLR framework, SimCLR with SAM, and SimCLR with the self-attention layer. The experimental results demonstrate the superior performance of our approach, especially in semi-supervised and transfer learning scenarios. It outperforms existing models, marking a significant advancement in using self-supervised learning to extract valuable insights from unlabeled ambient sensor data in real-world environments.

show abstract

Section: Simclr For Human Activity Recognitionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Enhancing Human Activity Recognition in Smart Homes with Self-Supervised Learning and Self-Attention

Chen,

Gouin-Vallerand,

Bouchard

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

“…Kermiche [18] introduced a contrastive Hebbian feedforward learning scheme for Boltzmann machines, which could be employed to improve the training of deep neural network based on the estimation only based on feedforward computations, local contrastive Hebbian correlations, and local disturbances. Wang et al [19] integrated the clustering scheme into the contrastive learning framework for human activity recognition, which could select the same-cluster samples from negative pairs based on a newly defined contrastive loss function. Zhu et al [20] combined the reinforcement learning and contrastive learning together, and addressed a multi-instance reinforcement contrastive learning framework, in which a reinforcement learning-based agent was designed to assist the contrastive learning via better selection of the discriminative feature sets with inherent semantic relationships.…”

Section: B Contrastive Learningmentioning

confidence: 99%

Personalized Federated Learning With Model-Contrastive Learning for Multi-Modal User Modeling in Human-Centric Metaverse

Zhou,

Yang,

Zheng

et al. 2024

IEEE J. Select. Areas Commun.

View full text Add to dashboard Cite

With the flourish of digital technologies and rapid development of 5G and beyond networks, Metaverse has become an increasingly hotly discussed topic, which offers users with multiple roles for diversified experience interacting with virtual services. How to capture and model users' multi-platform or cross-space data/behaviors become essential to enrich people with more realistic and immersed experience in Metaverseenabled smart applications over 5G and beyond networks. In this study, we propose a Personalized Federated Learning with Model-Contrastive Learning (PFL-MCL) framework, which may efficiently enhance the communication and interaction in humancentric Metaverse environments by making use of the largescale, heterogeneous, and multi-modal Metaverse data. Differing from the conventional Federated Learning (FL) architecture, a multi-center aggregation structure to learn multiple global models based on the changes of dynamically updated local model weights, is developed in global, while a hierarchical neural network structure which includes a personalized module and a federated module to tackle both issues on data heterogeneity and model heterogeneity, is designed in local, so as to enhance the performance of PFL with unique characteristics of Metaverse data. In particular, a two-stage iterative clustering algorithm with a more precise initialization is developed to facilitate the personalized global aggregation with dynamically updated multiple aggregation centers. A personalized multi-modal fusion network is constructed to greatly reduce the computational cost and feature dimensions from the high-dimensional heterogeneous inputs for more efficient cross-modal fusion, based on a hierarchical shift-window attention mechanism and a newly designed bridge attention mechanism. A MCL scheme is then incorporated to speed up the model convergence with less communication

show abstract

“…However, it is challenging to find an augmentation approach that performs well across all sensor datasets in the HAR field [35]. Furthermore, the paper [52] found that augment-positives in contrastive loss (see (6)) classify each activity instance into a single class in the latent space, thereby decreasing the same-class instance aggregations. Therefore, to ensure the generalization of our model, we remove both the consistency loss and augmented-positives.…”

Section: Reasons For Two Reductionsmentioning

confidence: 99%

More Reliable Neighborhood Contrastive Learning for Novel Class Discovery in Sensor-Based Human Activity Recognition

Zhang,

Zhu,

Nie

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

Human Activity Recognition (HAR) systems have made significant progress in recognizing and classifying human activities using sensor data from a variety of sensors. Nevertheless, they have struggled to automatically discover novel activity classes within massive amounts of unlabeled sensor data without external supervision. This restricts their ability to classify new activities of unlabeled sensor data in real-world deployments where fully supervised settings are not applicable. To address this limitation, this paper presents the Novel Class Discovery (NCD) problem, which aims to classify new class activities of unlabeled sensor data by fully utilizing existing activities of labeled data. To address this problem, we propose a new end-to-end framework called More Reliable Neighborhood Contrastive Learning (MRNCL), which is a variant of the Neighborhood Contrastive Learning (NCL) framework commonly used in visual domain. Compared to NCL, our proposed MRNCL framework is more lightweight and introduces an effective similarity measure that can find more reliable k-nearest neighbors of an unlabeled query sample in the embedding space. These neighbors contribute to contrastive learning to facilitate the model. Extensive experiments on three public sensor datasets demonstrate that the proposed model outperforms existing methods in the NCD task in sensor-based HAR, as indicated by the fact that our model performs better in clustering performance of new activity class instances.

show abstract

Negative Selection by Clustering for Contrastive Learning in Human Activity Recognition

Cited by 16 publications

References 39 publications

Enhancing Human Activity Recognition in Smart Homes with Self-Supervised Learning and Self-Attention

Enhancing Human Activity Recognition in Smart Homes with Self-Supervised Learning and Self-Attention

Personalized Federated Learning With Model-Contrastive Learning for Multi-Modal User Modeling in Human-Centric Metaverse

More Reliable Neighborhood Contrastive Learning for Novel Class Discovery in Sensor-Based Human Activity Recognition

Contact Info

Product

Resources

About