Contrastive Learning for Online Semi-Supervised General Continual Learning

Marilleau, Nicolas; Negrel, Romain; Chierchia, Giovanni; Bercher, Jean-Fmncois

doi:10.1109/icip46576.2022.9897290

Cited by 1 publication

(2 citation statements)

References 24 publications

(29 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For clear variants, each task is composed of non-overlapping classes while in blurry variants we introduce some overlap with the procedure previously described. More details are given in Appendix (Michel et al 2023).…”

Section: Evaluation Protocolmentioning

confidence: 99%

See 1 more Smart Citation

Learning Representations on the Unit Sphere: Investigating Angular Gaussian and Von Mises-Fisher Distributions for Online Continual Learning

Michel,

Chierchia,

Negrel

et al. 2024

AAAI

View full text Add to dashboard Cite

We use the maximum a posteriori estimation principle for learning representations distributed on the unit sphere. We propose to use the angular Gaussian distribution, which corresponds to a Gaussian projected on the unit-sphere and derive the associated loss function. We also consider the von Mises-Fisher distribution, which is the conditional of a Gaussian in the unit-sphere. The learned representations are pushed toward fixed directions, which are the prior means of the Gaussians; allowing for a learning strategy that is resilient to data drift. This makes it suitable for online continual learning, which is the problem of training neural networks on a continuous data stream, where multiple classification tasks are presented sequentially so that data from past tasks are no longer accessible, and data from the current task can be seen only once. To address this challenging scenario, we propose a memory-based representation learning technique equipped with our new loss functions. Our approach does not require negative data or knowledge of task boundaries and performs well with smaller batch sizes while being computationally efficient. We demonstrate with extensive experiments that the proposed method outperforms the current state-of-the-art methods on both standard evaluation scenarios and realistic scenarios with blurry task boundaries. For reproducibility, we use the same training pipeline for every compared method and share the code at https://github.com/Nicolas1203/ocl-fd.

show abstract

Section: Evaluation Protocolmentioning

confidence: 99%

“…For OCM we used the parameters from the original paper. Details regarding parameter selection can be found in Appendix (Michel et al 2023). 1: Final average accuracy (%) for all methods on datasets CIFAR10 split into 5 tasks, CIFAR100 split into 10 tasks, and TinyIN split into 100 tasks for varying memory sizes M .…”

Section: Evaluation Protocolmentioning

confidence: 99%