DeeSIL: Deep-Shallow Incremental Learning

Belouadah, Eden; Popescu, Adrian

doi:10.1007/978-3-030-11012-3_11

Cited by 40 publications

(49 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The performance exhibited by our approach is comparable to that of generative methods, while we rely on less restrictive hypotheses than generative methods-since they need to have information about all unseen classes to generate the samples required to learn the discriminative models. An alternative with the generative approach is to use incremental learning systems [29,5] but it usually leads to a significant drop in performance. Hence, our proposal has a practical interest for real systems that aim to recognize unseen classes whose number increases regularly.…”

Section: Discussionmentioning

confidence: 99%

Modeling Inter and Intra-Class Relations in the Triplet Loss for Zero-Shot Learning

Cacheux

Borgne

Crucianu

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Recognizing visual unseen classes, i.e. for which no training data is available, is known as Zero Shot Learning (ZSL). Some of the best performing methods apply the triplet loss to seen classes to learn a mapping between visual representations of images and attribute vectors that constitute class prototypes. They nevertheless make several implicit assumptions that limit their performance on real use cases, particularly with fine-grained datasets comprising a large number of classes. We identify three of these assumptions and put forward corresponding novel contributions to address them. Our approach consists in taking into account both inter-class and intra-class relations, respectively by being more permissive with confusions between similar classes, and by penalizing visual samples which are atypical to their class. The approach is tested on four datasets, including the large-scale ImageNet, and exhibits performances significantly above recent methods, even generative methods based on more restrictive hypotheses.

show abstract

Section: Discussionmentioning

confidence: 99%

Modeling Inter and Intra-Class Relations in the Triplet Loss for Zero-Shot Learning

Cacheux

Borgne

Crucianu

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

show abstract

“…Incremental learning methods attempt to follow the human cognition system, that is, learn concepts sequentially [32]. According to gradually extending acquired knowledge, incremental learning methods can adapt the data streams from different domains or tasks.…”

Section: B Incremental Learningmentioning

confidence: 99%

Incremental SAR Automatic Target Recognition With Error Correction and High Plasticity

Tang

Xiang

Zhang

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Synthetic aperture radar automatic target recognition (SAR ATR) uses computer processing capabilities to infer the classes of the targets without human intervention. For SAR ATR, deep learning gradually emerges as a powerful tool and achieves promising performance. However, it faces serious challenges of how to deal with incremental recognition scenarios. The existing deep learning-based SAR ATR methods usually predefine the total number of recognition classes. In realistic applications, the new tasks/classes will be added continuously. If all old data are stored and mixed with newly added data to update the model, the storage pressure and time consumption make the application infeasible. In this article, the high plastic error correction incremental learning (HPecIL) is proposed to address the model degradation and plasticity decline in the incremental scenario. Multiple optimal models trained on old tasks are used to correct accumulative errors and alleviate model degradation. Moreover, the sharp data distribution shift due to newly added data can also result in the model underperforming. A classbalanced training batch is constructed to deal with the issue of unbalanced data distribution. To make a trade-off between model stability and model plasticity, low-effect nodes in the model are removed to boost the efficiency of model update. The proposed HPecIL outperforms the other state-of-the-art methods in incremental recognition scenarios. The experimental results demonstrate the effectiveness of the proposed method.

show abstract

“…We study the common continual learning paradigm in which pre-training precedes continual learning [28,29,55,10,71,32,33,34,68,4]. Formally, given a pre-training dataset {(X i , y i )} ∈ A, with M images X i and their corresponding labels y i ∈ Y, a set of parameters θ are learned for a CNN using A in an offline manner, i.e., the learner can shuffle the data to simulate independent and identically distributed data and loop over it as many times as it desires.…”

Section: Problem Formulationmentioning

confidence: 99%

“…For all experiments, we use ResNet-18 as the CNN. For continual learning with the full-resolution ImageNet dataset, ResNet-18 has been adopted as the universal standard CNN architecture by the community [55,68,28,32,21,28,10,5,4].…”

Section: Algorithmsmentioning

confidence: 99%

Self-Supervised Training Enhances Online Continual Learning

Jhair¹,

Hayes²,

Kanan³

2021

Preprint

View full text Add to dashboard Cite

In continual learning, a system must incrementally learn from a non-stationary data stream without catastrophic forgetting. Recently, multiple methods have been devised for incrementally learning classes on large-scale image classification tasks, such as ImageNet. State-of-the-art continual learning methods use an initial supervised pre-training phase, in which the first 10% -50% of the classes in a dataset are used to learn representations in an offline manner before continual learning of new classes begins. We hypothesize that self-supervised pre-training could yield features that generalize better than supervised learning, especially when the number of samples used for pre-training is small. We test this hypothesis using the self-supervised MoCo-V2 and SwAV algorithms. On ImageNet, we find that both outperform supervised pre-training considerably for online continual learning, and the gains are larger when fewer samples are available. Our findings are consistent across three continual learning algorithms. Our best system achieves a 14.95% relative increase in top-1 accuracy on class incremental ImageNet over the prior state of the art for online continual learning.

show abstract

DeeSIL: Deep-Shallow Incremental Learning

Cited by 40 publications

References 17 publications

Modeling Inter and Intra-Class Relations in the Triplet Loss for Zero-Shot Learning

Modeling Inter and Intra-Class Relations in the Triplet Loss for Zero-Shot Learning

Incremental SAR Automatic Target Recognition With Error Correction and High Plasticity

Self-Supervised Training Enhances Online Continual Learning

Contact Info

Product

Resources

About