Balancing Stability and Plasticity through Advanced Null Space in Continual Learning

Abstract

Continual learning is a learning paradigm that learns tasks sequentially with resources constraints, in which the key challenge is stability-plasticity dilemma, i.e., it is uneasy to simultaneously have the stability to prevent catastrophic forgetting of old tasks and the plasticity to learn new tasks well. In this paper, we propose a new continual learning approach, Advanced Null Space (AdNS), to balance the stability and plasticity without storing any old data of previous tasks. Specifically, to obtain better stability, AdNS makes use of low-rank approximation to obtain a novel null space and projects the gradient onto the null space to prevent the interference on the past tasks. To control the generation of the null space, we introduce a non-uniform constraint strength to further reduce forgetting. Furthermore, we present a simple but effective method, intra-task distillation, to improve the performance of the current task. Finally, we theoretically find that null space plays a key role in plasticity and stability, respectively. Experimental results show that the proposed method can achieve better performance compared to state-ofthe-art continual learning approaches.

Restricted Orthogonal Gradient Projection for Continual Learning

“…NCL (Kao et al, 2021) combines the idea of gradient projection and Bayesian weight regularization to mitigate catastrophic forgetting. In spite of minimizing backward interference, these approaches suffer poor forward knowledge transfer and lack plasticity (Kong et al, 2022). TRGP (Lin et al, 2022) expands the model with trust regions to achieve better performance on new tasks by introducing additional scale parameters.…”

“…To mitigate forgetting and facilitate forward knowledge transfer, replay-based methods (Lopez-Paz & Ranzato, 2017;Shin et al, 2017;Choi et al, 2021) stores some old samples in the memory, and expansion-based methods (Rusu et al, 2016;Yoon et al, 2017;2019) expand the model structure to accommodate incoming knowledge. However, these methods require either extra memory buffers (Parisi et al, 2019) or a growing network architecture as new tasks continually arrive (Kong et al, 2022), which are always computationally expansive (De Lange et al, 2021). Thus, promoting performance within a fixed network capacity remains challenging.…”