New Insights on Reducing Abrupt Representation Change in Online Continual Learning

Abstract

We study the online continual learning paradigm, where agents must learn from a changing distribution with constrained memory and compute. Previous work often tackle catastrophic forgetting by overcoming changes in the space of model parameters. In this work we instead focus on the change in representations of previously observed data due to the introduction of previously unobserved class samples in the incoming data stream. We highlight the issues that arise in the practical setting where new classes must be distinguished between all previous classes. Starting from a popular approach, experience replay, we consider a metric learning based loss function, the triplet loss, which allows us to more explicitly constrain the behavior of representations. We hypothesize and empirically confirm that the selection of negatives used in the triplet loss plays a major role in the representation change, or drift, of previously observed data and can be greatly reduced by appropriate negative selection. Motivated by this we further introduce a simple adjustment to the standard cross entropy loss used in prior experience replay that achieves similar effect. Our approach greatly improves the performance of experience replay and obtains state-of-the-art on several existing benchmarks in online continual learning, while remaining efficient in both memory and compute.

Class-Incremental Continual Learning into the eXtended DER-verse

“…On the one hand, the negative bias towards past classes can be ascribed to the optimization of the cross-entropy loss on examples from the current task. As pointed out in [28], when a new task is presented to the net, an asymmetry arises between the contributions of replay data and current examples to the weights updates: indeed, the gradients of new (and poorly fit) examples outweigh (Fig. 2b).…”

“…As also observed in other recent works [28], [29], [42], this issue can be mitigated by revising the way the cross-entropy loss is applied during training. Given an example from the current task, we avoid computing the softmax activation on all logits and instead restrict it on those modeling the scores of the current task classes.…”

“…As done in [11], we use LwF.MC (the CiCL adaptation of LwF); • Experience Replay (ER) [33], [34] is a simple rehearsal method that stores previously encountered examples in a memory buffer for later replay. In spite of its simplicity, it still stands as a strong baseline [15], [18]; • ER with Asymmetric Cross-Entropy (ER-ACE) [28] is a recently proposed modification of ER that addresses stream imbalance w.r.t. to the memory buffer by optimizing separate cross-entropy loss terms; • Incremental Classifier and Representation Learning (iCaRL) [11] combines a carefully-designed herding buffer with a nearest mean-of-exemplars classifier; it is often regarded as a strong performer on complex datasets; • Bias Correction (BiC) [41] pairs Experience Replay with a regularization term that resembles the objective of LwF.…”

“…By contrast, recent CL works focus mainly on Class-IL [12], [28], [29], [30], [31], as it is more general and regarded as more realistic than the other scenarios [12], [32].…”

“…Several works [28], [29], [41], [42] have recently shed light on the accumulation of bias towards present classes and the negative impact it has on performance. We have found that also DER(++) are prone to such a pitfall: similarly to [41], we can quantitatively characterize it by evaluating how predictions distribute across different classification heads (as training progresses).…”