Continual Learning through Retrieval and Imagination

Abstract

Continual learning is an intellectual ability of artificial agents to learn new streaming labels from sequential data. The main impediment to continual learning is catastrophic forgetting, a severe performance degradation on previously learned tasks. Although simply replaying all previous data or continuously adding the model parameters could alleviate the issue, it is impractical in real-world applications due to the limited available resources. Inspired by the mechanism of the human brain to deepen its past impression, we propose a novel framework, Deep Retrieval and Imagination (DRI), which consists of two components: 1) an embedding network that constructs a unified embedding space without adding model parameters on the arrival of new tasks; and 2) a generative model to produce additional (imaginary) data based on the limited memory. By retrieving the past experiences and corresponding imaginary data, DRI distills knowledge and rebalances the embedding space to further mitigate forgetting. Theoretical analysis demonstrates that DRI can reduce the loss approximation error and improve the robustness through retrieval and imagination, bringing better generalizability to the network. Extensive experiments show that DRI performs significantly better than the existing state-of-the-art continual learning methods and effectively alleviates catastrophic forgetting.

Neuron importance algorithm for continual learning

“…Replay methods preserve the sample information from old tasks and utilize it for the training process of new tasks to mitigate catastrophic forgetting [10][11][12][13][14][15][16][17][18][19][20][21][22] . iCaRL 15 selects some representative samples from old datasets and adds them to the dataset of new tasks.…”

Leveraging Old Knowledge to Continually Learn New Classes in Medical Images

“…Data replay-based methods interleaves data related to previous tasks with new data so that the model is 'reminded' of the old knowledge. One way to obtain the old data is by training deep generative models to synthesize 'fake' samples that mimics data from previous classes (Shin et al 2017;Wu et al 2018;Rao et al 2019;Wang et al 2022) but there is the issue of generating realistic data samples. Another way is to directly store a small number of previous class samples to train together with the new data (Rebuffi et al 2017;Chaudhry et al 2019).…”

Adaptive Self-Supervised Continual Learning

“…Continual Learning (CL) studies the problem of learning new knowledge incrementally while preserving previously learned knowledge. Following [10], CL methods can be divided into three groups: rehearsal-based methods [30,44,36,3,37,11], regularizationbased methods [18,20,22,19,42], and parameter isolation methods [31,32,28]. However, the majority of existing CL methods focus on supervised settings while Continual Self-Supervised Continual Learning (CSSL) is surprisingly under-investigated.…”