Investigating Enhancements to Contrastive Predictive Coding for Human Activity Recognition

Haresamudram, Harish; Essa, Irfan; Plötz, Thomas

doi:10.1109/percom56429.2023.10099197

Cited by 4 publications

(14 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As such, self‐supervision, that is, the paradigm of “pretrain‐then‐finetune” has the potential to become a strong alternative to the previously dominant end‐to‐end training approaches. Recently a number of contributions have been made that introduced self‐supervised learning into the field of HAR, including benchmarks of leading techniques (Haresamudram, Essa, and Plötz 2022) and novel computational methods (Haresamudram, Essa, and Plötz 2023).…”

Section: Personalized Longitudinal Interactionmentioning

confidence: 99%

AI‐CARING: National AI Institute for Collaborative Assistance and Responsive Interaction for Networked Groups

Chernova,

Mynatt,

Rozga

et al. 2024

AI Magazine

View full text Add to dashboard Cite

Over 13 million Americans aged 65 and older are currently living with a diagnosis of mild cognitive impairment (MCI), a common precursor to dementia. These individuals largely rely on a network of informal caregivers—family, friends, and community members—who work together with professional healthcare and social service providers to provide care and support in home settings. The AI‐CARING Institute contributes foundational AI research focused on developing personalized collaborative AI systems that improve the quality of life and independence of aging adults living at home.

show abstract

Section: Personalized Longitudinal Interactionmentioning

confidence: 99%

AI‐CARING: National AI Institute for Collaborative Assistance and Responsive Interaction for Networked Groups

Chernova,

Mynatt,

Rozga

et al. 2024

AI Magazine

View full text Add to dashboard Cite

show abstract

“…SimSiam employs a pair of mirror-image networks and a predictor network at the end of a node [62][63][64]. The loss function employs an asymmetric stop gradient to optimize the pairwise alignments between positive pairs because the two branches have identical weights.…”

Section: Non-contrastive Ssl Modelsmentioning

confidence: 99%

Consequential Advancements of Self-Supervised Learning (SSL) in Deep Learning Contexts

Abdulrazzaq,

Ramaha,

Hameed

et al. 2024

Mathematics

View full text Add to dashboard Cite

Self-supervised learning (SSL) is a potential deep learning (DL) technique that uses massive volumes of unlabeled data to train neural networks. SSL techniques have evolved in response to the poor classification performance of conventional and even modern machine learning (ML) and DL models of enormous unlabeled data produced periodically in different disciplines. However, the literature does not fully address SSL’s practicalities and workabilities necessary for industrial engineering and medicine. Accordingly, this thorough review is administered to identify these prominent possibilities for prediction, focusing on industrial and medical fields. This extensive survey, with its pivotal outcomes, could support industrial engineers and medical personnel in efficiently predicting machinery faults and patients’ ailments without referring to traditional numerical models that require massive computational budgets, time, storage, and effort for data annotation. Additionally, the review’s numerous addressed ideas could encourage industry and healthcare actors to take SSL principles into an agile application to achieve precise maintenance prognostics and illness diagnosis with remarkable levels of accuracy and feasibility, simulating functional human thinking and cognition without compromising prediction efficacy.

show abstract

“…[78] examines the pre-training data efficiency, i.e., the minimal quantities of pre-training data required for effective wearablebased self-supervised learning. Enhancements to wearable-based CPC were investigated in [30], by considering three components: the encoder architecture, the autoregressive network, and the future timestep prediction task. The resulting 'Enhanced CPC' demonstrates substantial improvements over the original framework [68] as well as outperforms state-of-the-art self-supervision on four of six target datasets.…”

Section: Self-supervised Representation Learning For Human Activity R...mentioning

confidence: 99%

“…Following the self-supervised learning paradigm, our approach contains two stages: (i) pre-training, where the network learns to map unlabeled data to a codebook of vectors, resulting in the discrete representations; and (ii) fine-tuning/classification, which utilizes the discrete representations as input for recognizing activities. In order to enable the mapping, we apply vector quantization (VQ) to the Enhanced CPC framework [30]. Therefore, the base of the discretization process is self-supervision, where the loss from the pretext task is added to the loss from the VQ module in order to update the network parameters as well as the codebook vectors.…”

Section: Input Window Discretizationmentioning

confidence: 99%

“…To this end, we apply learned vector quantization (VQ) [27,29] to wearables applications, which enables us to directly learn the mapping between short spans of sensor data and a codebook of vectors, where the index of the closest codebook vector comprises the discrete representation. In addition, we utilize self-supervised learning as a base (via the Enhanced CPC framework [30]), thereby deriving the representations without the need for annotations. Sensor data are first encoded using convolutional blocks, which can handle multiple data channels (e.g., x-y-z axes of triaxial accelerometry) in a straightforward manner.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Towards Learning Discrete Representations via Self-Supervision for Wearables-Based Human Activity Recognition

Haresamudram,

Essa,

Plötz

2024

Sensors

Self Cite

View full text Add to dashboard Cite

Human activity recognition (HAR) in wearable and ubiquitous computing typically involves translating sensor readings into feature representations, either derived through dedicated pre-processing procedures or integrated into end-to-end learning approaches. Independent of their origin, for the vast majority of contemporary HAR methods and applications, those feature representations are typically continuous in nature. That has not always been the case. In the early days of HAR, discretization approaches had been explored—primarily motivated by the desire to minimize computational requirements on HAR, but also with a view on applications beyond mere activity classification, such as, for example, activity discovery, fingerprinting, or large-scale search. Those traditional discretization approaches, however, suffer from substantial loss in precision and resolution in the resulting data representations with detrimental effects on downstream analysis tasks. Times have changed, and in this paper, we propose a return to discretized representations. We adopt and apply recent advancements in vector quantization (VQ) to wearables applications, which enables us to directly learn a mapping between short spans of sensor data and a codebook of vectors, where the index comprises the discrete representation, resulting in recognition performance that is at least on par with their contemporary, continuous counterparts—often surpassing them. Therefore, this work presents a proof of concept for demonstrating how effective discrete representations can be derived, enabling applications beyond mere activity classification but also opening up the field to advanced tools for the analysis of symbolic sequences, as they are known, for example, from domains such as natural language processing. Based on an extensive experimental evaluation of a suite of wearable-based benchmark HAR tasks, we demonstrate the potential of our learned discretization scheme and discuss how discretized sensor data analysis can lead to substantial changes in HAR.

show abstract

Investigating Enhancements to Contrastive Predictive Coding for Human Activity Recognition

Cited by 4 publications

References 38 publications

AI‐CARING: National AI Institute for Collaborative Assistance and Responsive Interaction for Networked Groups

AI‐CARING: National AI Institute for Collaborative Assistance and Responsive Interaction for Networked Groups

Consequential Advancements of Self-Supervised Learning (SSL) in Deep Learning Contexts

Towards Learning Discrete Representations via Self-Supervision for Wearables-Based Human Activity Recognition

Contact Info

Product

Resources

About