Self-Supervised Video Representation Learning with Meta-Contrastive Network

Lin, Yuanze; Guo, Xianping; Lu, Yan

doi:10.1109/iccv48922.2021.00813

Cited by 32 publications

(5 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While our work focused on supervised learning with visual data, our method and general approach is applicable to other supervised learning tasks. Indeed, the MAML framework has been established for tasks such as reinforcement learning (46), multilingual speech emotion classiőcation (47), and self-supervised learning (48). In this work, SOEL was limited to the őnal layer of the network.…”

Section: Discussionmentioning

confidence: 99%

Emulating Brain-like Rapid Learning in Neuromorphic Edge Computing

Stewart,

Neumeier,

Shrestha

et al. 2024

Preprint

View full text Add to dashboard Cite

Achieving personalized intelligence at the edge with real-time learning capabilities holds enormous promise to enhance our daily experiences and assist in decision-making, planning, and sensing. Yet, today's technology encounters difficulties with efficient and reliable learning at the edge, due to a lack of personalized data, insufficient hardware capabilities, and the inherent challenges posed by online learning. Over time and across multiple developmental phases, the brain has evolved to efficiently incorporate new knowledge by gradually building on previous knowledge. In this work, we emulate this process in digital neuromorphic technology emulating the neural and synaptic processes of the brain by means of two stages of learning. Initially, a meta-training phase trains the learning hardware's hyperparameters for one-shot learning by deploying a differentiable simulation of local three-factor synaptic plasticity rules implemented in the neuromorphic chip. This meta-training process refines the synaptic plasticity and related hyperparameters to align with the specific dynamics inherent in the hardware and the given task domain. During the subsequent deployment stage, these optimized hyperparameters enable fast, data-efficient and accurate learning of new classes. We demonstrate our approach using event-driven vision sensor data and the Intel Loihi neuromorphic processor with its plasticity dynamics, achieving state-of-the-art accuracy on learning new categories in one-shot in real-time among three distinct task domains. Our methodology can be deployed with arbitrary plasticity models and can be applied to situations demanding quick learning and adaptation at the edge, such as navigating unfamiliar environments, or learning unexpected categories of data through user engagement.

show abstract

Section: Discussionmentioning

confidence: 99%

Emulating Brain-like Rapid Learning in Neuromorphic Edge Computing

Stewart,

Neumeier,

Shrestha

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…object recognition [29], video representation learning [54]. There are also some prior works in continual learning.…”

Section: Related Workmentioning

confidence: 99%

Online Continual Learning via the Meta-learning update with Multi-scale Knowledge Distillation and Data Augmentation

Han

Liu

2022

Engineering Applications of Artificial Intelligence

View full text Add to dashboard Cite

“…Conventional KD and SSL methods offer numerous potential teacher choices, e.g., larger but static pretrained teacher networks [27], or networks that share the same model architecture but use weights from a previous epoch [42], or as an exponential moving average [24]. The primary drawback to these approaches is reduced computational and memory efficiency as they require a secondary inference stage using additional model weights that must be kept in memory.…”

Section: Teacher Network Selectionmentioning

confidence: 99%

Robust Cross-Modal Representation Learning with Progressive Self-Distillation

Andonian¹,

Chen²,

Hamid³

2022

Preprint

View full text Add to dashboard Cite

The learning objective of vision-language approach of CLIP [63] does not effectively account for the noisy manyto-many correspondences found in web-harvested image captioning datasets, which contributes to its compute and data inefficiency. To address this challenge, we introduce a novel training framework based on cross-modal contrastive learning that uses progressive self-distillation and soft image-text alignments to more efficiently learn robust representations from noisy data. Our model distills its own knowledge to dynamically generate soft-alignment targets for a subset of images and captions in every minibatch, which are then used to update its parameters. Extensive evaluation across 14 benchmark datasets shows that our method consistently outperforms its CLIP counterpart in multiple settings, including: (a) zero-shot classification, (b) linear probe transfer, and (c) image-text retrieval, without incurring extra computational cost. Analysis using an ImageNet-based robustness test-bed [70] reveals that our method offers better effective robustness to natural distribution shifts compared to both ImageNet-trained models and CLIP itself. Lastly, pretraining with datasets spanning two orders of magnitude in size shows that our improvements over CLIP tend to scale with number of training examples.

show abstract

Self-Supervised Video Representation Learning with Meta-Contrastive Network

Cited by 32 publications

References 27 publications

Emulating Brain-like Rapid Learning in Neuromorphic Edge Computing

Emulating Brain-like Rapid Learning in Neuromorphic Edge Computing

Online Continual Learning via the Meta-learning update with Multi-scale Knowledge Distillation and Data Augmentation

Robust Cross-Modal Representation Learning with Progressive Self-Distillation

Contact Info

Product

Resources

About