Adaptive Aggregation Networks for Class-Incremental Learning

Liu, Yaoyao; Schiele, Bernt; Sun, Quansen

doi:10.1109/cvpr46437.2021.00257

Cited by 138 publications

(132 citation statements)

References 50 publications

Supporting

Mentioning

108

Contrasting

Order By: Relevance

“…Baseline: We evaluate representative memory replay approaches such as LwF (Li & Hoiem, 2017), iCaRL (Rebuffi et al, 2017, BiC (Wu et al, 2019), LUCIR (Hou et al, 2019), Mnemonics (Liu et al, 2020b), TPCIL (Tao et al, 2020a), PODNet (Douillard et al, 2020), DDE (Hu et al, 2021) and AANets (Liu et al, 2021a). In particular, AANets and DDE are the recent strong approaches implemented on the backbones of LUCIR and PODNet, so we also implement ours on the two backbones.…”

Section: Methodsmentioning

confidence: 99%

“…Replay-based methods (Rebuffi et al, 2017;Shin et al, 2017) approximated and recovered the old data distribution. In particular, memory replay of representative old training samples (referred to as memory replay) can generally achieve the best performance in class-incremental learning (Liu et al, 2021a;Hu et al, 2021) and in numerous other continual learning scenarios, such as audio tasks (Ehret et al, 2020), few-shot (Tao et al, 2020b), semi-supervised (Wang et al, 2021a), and unsupervised continual learning (Khare et al, 2021).…”

Section: Related Workmentioning

confidence: 99%

“…To better construct the memory buffer, Mnemonics (Liu et al, 2020b) parameterized the original data and made them optimizable, while TPCIL (Tao et al, 2020a) constructed an elastic Hebbian graph by competitive Hebbian learning. On the other hand, BiC (Wu et al, 2019), LUCIR (Hou et al, 2019), PODNet (Douillard et al, 2020), DDE (Hu et al, 2021) and AANets (Liu et al, 2021a) attempted to better exploit the memory buffer, such as by mitigating the data imbalance between old and new classes (Hou et al, 2019;Wu et al, 2019;Hu et al, 2021).…”

Section: Related Workmentioning

confidence: 99%

“…However, due to the low storage efficiency of saving original data, this strategy of building memory buffer will lose a lot of information about the old data distribution. On the other hand, this usually requires huge extra computation to further mitigate catastrophic forgetting, such as by learning additional parameters (Liu et al, 2021a) or distilling old features (Hu et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Memory Replay with Data Compression for Continual Learning

Wang¹,

Zhang²,

Kuo³

et al. 2022

Preprint

View full text Add to dashboard Cite

Continual learning needs to overcome catastrophic forgetting of the past. Memory replay of representative old training samples has been shown as an effective solution, and achieves the state-of-the-art (SOTA) performance. However, existing work is mainly built on a small memory buffer containing a few original data, which cannot fully characterize the old data distribution. In this work, we propose memory replay with data compression to reduce the storage cost of old training samples and thus increase their amount that can be stored in the memory buffer. Observing that the trade-off between the quality and quantity of compressed data is highly nontrivial for the efficacy of memory replay, we propose a novel method based on determinantal point processes (DPPs) to efficiently determine an appropriate compression quality for currently-arrived training samples. In this way, using a naive data compression algorithm with a properly selected quality can largely boost recent strong baselines by saving more compressed data in a limited storage space. We extensively validate this across several benchmarks of class-incremental learning and in a realistic scenario of object detection for autonomous driving.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Memory Replay with Data Compression for Continual Learning

Wang¹,

Zhang²,

Kuo³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Class Incremental Learning (CIL). Incremental learning aims to continuously learn by accumulating past knowledge [2,19,24]. Our work is conducted on CIL benchmarks, which need to learn a unified classifier that can recognize all the old and new classes combined.…”

Section: Related Workmentioning

confidence: 99%

Distilling Causal Effect of Data in Class-Incremental Learning

Hu¹,

Tang²,

Chen³

et al. 2021

Preprint

View full text Add to dashboard Cite

We propose a causal framework to explain the catastrophic forgetting in Class-Incremental Learning (CIL) and then derive a novel distillation method that is orthogonal to the existing anti-forgetting techniques, such as data replay and feature/label distillation. We first 1) place CIL into the framework, 2) answer why the forgetting happens: the causal effect of the old data is lost in new training, and then 3) explain how the existing techniques mitigate it: they bring the causal effect back. Based on the framework, we find that although the feature/label distillation is storageefficient, its causal effect is not coherent with the end-toend feature learning merit, which is however preserved by data replay. To this end, we propose to distill the Colliding Effect between the old and the new data, which is fundamentally equivalent to the causal effect of data replay, but without any cost of replay storage. Thanks to the causal effect analysis, we can further capture the Incremental Momentum Effect of the data stream, removing which can help to retain the old effect overwhelmed by the new data effect, and thus alleviate the forgetting of the old class in testing. Extensive experiments on three CIL benchmarks: CIFAR-100, ImageNet-Sub&Full, show that the proposed causal effect distillation can improve various state-of-the-art CIL methods by a large margin (0.72%-9.06%). 1 1 There are also other settings like task-incremental [14,26].

show abstract

Pyramid Self-attention for Semantic Segmentation

Wang

et al. 2021

Pattern Recognition and Computer Vision

View full text Add to dashboard Cite

Continually learning to segment more and more types of image regions is a desired capability for many intelligent systems. However, such continual semantic segmentation suffers from the same catastrophic forgetting issue as in continual classification learning. While multiple knowledge distillation strategies originally for continual classification have been well adapted to continual semantic segmentation, they only consider transferring old knowledge based on the outputs from one or more layers of deep fully convolutional networks. Different from existing solutions, this study proposes to transfer a new type of information relevant to knowledge, i.e. the relationships between elements (Eg. pixels or small local regions) within each image which can capture both within-class and between-class knowledge. The relationship information can be effectively obtained from the self-attention maps in a Transformer-style segmentation model. Considering that pixels belonging to the same class in each image often share similar visual properties, a class-specific region pooling is applied to provide more efficient relationship information for knowledge transfer. Extensive evaluations on multiple public benchmarks support that the proposed self-attention transfer method can further effectively alleviate the catastrophic forgetting issue, and its flexible combination with one or more widely adopted strategies significantly outperforms state-of-the-art solutions.

show abstract

Adaptive Aggregation Networks for Class-Incremental Learning

Cited by 138 publications

References 50 publications

Memory Replay with Data Compression for Continual Learning

Memory Replay with Data Compression for Continual Learning

Distilling Causal Effect of Data in Class-Incremental Learning

Pyramid Self-attention for Semantic Segmentation

Contact Info

Product

Resources

About