Continuum: Simple Management of Complex Continual Learning Scenarios

Douillard, Arthur; Lesort, Timothée

doi:10.48550/arxiv.2102.06253

Cited by 4 publications

(4 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, creating diverse benchmarks, as well as approaches that do not critically rely on the assumptions from the default scenario, should be an ongoing effort. This effort should be pushed notably by existing continual learning libraries such as Continuum [12], Avalanche [13] or Sequoia [7].…”

Section: Sub-task Onsetsmentioning

confidence: 99%

Tutorial - Continual Learning beyond classification

Gepperth¹,

Lesort²

2022

ESANN 2022 Proceedings

Self Cite

View full text Add to dashboard Cite

Continual Learning (CL, sometimes also termed incremental learning) is a flavor of machine learning where the usual assumption of stationary data distribution is relaxed or omitted. When naively applying, e.g., DNNs in CL problems, changes in the data distribution can cause the so-called catastrophic forgetting (CF) effect: an abrupt loss of previous knowledge. Although many significant contributions to enabling CL have been made in recent years, most works address supervised (classification) problems. This article reviews literature that study CL in other settings, such as learning with reduced supervision, fully unsupervised learning, and reinforcement learning. Besides proposing a simple schema for classifying CL approaches w.r.t. their level of autonomy and supervision, we discuss the specific challenges associated with each setting and the potential contributions to the field of CL in general. Framework and Goals of Continual LearningContinual learning (CL) is a machine learning sub-field that studies learning under time-varying data distributions. This relaxes one of the fundamental assumptions of statistical learning theory ([1]), which states that the data follows 67

show abstract

Section: Sub-task Onsetsmentioning

confidence: 99%

Tutorial - Continual Learning beyond classification

Gepperth¹,

Lesort²

2022

ESANN 2022 Proceedings

Self Cite

View full text Add to dashboard Cite

show abstract

“…Working towards the standardization of implementations: Research in CIOD suffers from poor standardization and has not fully adopted the advent already developed by the CL community for reproducibility, such as the Avalanche and Continuum libraries [104,105]. Besides that, there is no standard implementation for most of the discussed solutions to leverage fair comparisons.…”

Section: Trends and Research Directionsmentioning

confidence: 99%

Continual Object Detection: A review of definitions, strategies, and challenges

Menezes¹,

Moura²,

Alves³

et al. 2022

Preprint

View full text Add to dashboard Cite

The field of Continual Learning investigates the ability to learn consecutive tasks without losing performance on those previously learned. Its focus has been mainly on incremental classification tasks. We believe that research in continual object detection deserves even more attention due to its vast range of applications in robotics and autonomous vehicles. This scenario is more complex than conventional classification given the occurrence of instances of classes that are unknown at the time, but can appear in subsequent tasks as a new class to be learned, resulting in missing annotations and conflicts with the background label. In this review, we analyze the current strategies proposed to tackle the problem of class-incremental object detection. Our main contributions are:(1) a short and systematic review of the methods that propose solutions to traditional incremental object detection scenarios; (2) A comprehensive evaluation of the existing approaches using a new metric to quantify the stability and plasticity of each technique in a standard way; (3) an overview of the current trends within continual object detection and a discussion of possible future research directions.

show abstract

“…There are 10 different classes, with each class represented in the dataset by 5 different objects that were each filmed in 11 different environments. As in [14,32], we use the images from eight of these environments for training and the others for testing. This results in approximately 10, 500 training images per class.…”

Section: Core50 With Pre-training On Imagenetmentioning

confidence: 99%

Class-Incremental Learning with Generative Classifiers

Ven¹,

Li²,

Tolias³

2021

Preprint

View full text Add to dashboard Cite

Incrementally training deep neural networks to recognize new classes is a challenging problem. Most existing class-incremental learning methods store data or use generative replay, both of which have drawbacks, while 'rehearsal-free' alternatives such as parameter regularization or bias-correction methods do not consistently achieve high performance. Here, we put forward a new strategy for class-incremental learning: generative classification. Rather than directly learning the conditional distribution p(y|x), our proposal is to learn the joint distribution p(x, y), factorized as p(x|y)p(y), and to perform classification using Bayes' rule. As a proof-of-principle, here we implement this strategy by training a variational autoencoder for each class to be learned and by using importance sampling to estimate the likelihoods p(x|y). This simple approach performs very well on a diverse set of continual learning benchmarks, outperforming generative replay and other existing baselines that do not store data.

show abstract

Continuum: Simple Management of Complex Continual Learning Scenarios

Cited by 4 publications

References 20 publications

Tutorial - Continual Learning beyond classification

Tutorial - Continual Learning beyond classification

Continual Object Detection: A review of definitions, strategies, and challenges

Class-Incremental Learning with Generative Classifiers

Contact Info

Product

Resources

About