Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffer

Smith, James Seale; Balloch, Jonathan; Hsu, Yen-Chang; Kira, Zsolt

doi:10.48550/arxiv.2101.09536

Cited by 8 publications

(8 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Avoiding privacy concerns, this work also follows a line of work that doesn't store real examples for experience replay, such as generating examples by GAN (Atkinson et al, 2018), synthesizing examples (Xu et al, 2022) by model-inversion (Smith et al, 2021b), and using unlabeled data in the learning environment (Smith et al, 2021a). In language domain, LAMOL (Sun et al, 2019) trains the language model to solve current tasks and generate current training examples simultaneously, then this model can generate "pseudo" old examples for replay before any new tasks.…”

Section: Related Workmentioning

confidence: 99%

Continual Sequence Generation with Adaptive Compositional Modules

Zhang¹,

Wang²,

Yang³

2022

Preprint

View full text Add to dashboard Cite

Continual learning is essential for real-world deployment when there is a need to quickly adapt the model to new tasks without forgetting knowledge of old tasks. Existing work on continual sequence generation either always reuses existing parameters to learn new tasks, which is vulnerable to catastrophic forgetting on dissimilar tasks, or blindly adds new parameters for every new task, which could prevent knowledge sharing between similar tasks. To get the best of both worlds, in this work, we propose continual sequence generation with adaptive compositional modules to adaptively add modules in transformer architectures and compose both old and new modules for new tasks. We also incorporate pseudo experience replay to facilitate knowledge transfer in those shared modules. Experiment results on various sequences of generation tasks show that our framework can adaptively add modules or reuse modules based on task similarity, outperforming state-of-the-art baselines in terms of both performance and parameter efficiency. We make our code public at https://github.com/GT-SALT/ Adaptive-Compositional-Modules.

show abstract

Section: Related Workmentioning

confidence: 99%

Continual Sequence Generation with Adaptive Compositional Modules

Zhang¹,

Wang²,

Yang³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In continual and lifelong learning there exists a sequence "tasks", each of which could be different environments, datasets, or novel classes. These tasks can be overlapping, task boundaries don't have to be welldefined, and can include a mixture of supervised and unsupervised data, but in most cases these tasks are disjoint and task boundaries are known and discrete (Parisi et al 2019;Silver, Yang, and Li 2013;Smith et al 2021). Most distinctly, in continual learning the model is trained on only one task at a time but validated on that task and all prior tasks.…”

Section: Novelty Background and Related Workmentioning

confidence: 99%

NovGrid: A Flexible Grid World for Evaluating Agent Response to Novelty

Balloch¹,

Lin²,

Hussain³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

A robust body of reinforcement learning techniques have been developed to solve complex sequential decision making problems. However, these methods assume that train and evaluation tasks come from similarly or identically distributed environments. This assumption does not hold in real life where small novel changes to the environment can make a previously learned policy fail or introduce simpler solutions that might never be found. To that end we explore the concept of novelty, defined in this work as the sudden change to the mechanics or properties of environment. We provide an ontology of for novelties most relevant to sequential decision making, which distinguishes between novelties that affect objects versus actions, unary properties versus non-unary relations, and the distribution of solutions to a task. We introduce NOVGRID, a novelty generation framework built on Mini-Grid, acting as a toolkit for rapidly developing and evaluating novelty-adaptation-enabled reinforcement learning techniques. Along with the core NOVGRIDwe provide exemplar novelties aligned with our ontology and instantiate them as novelty templates that can be applied to many MiniGridcompliant environments. Finally, we present a set of metrics built into our framework for the evaluation of noveltyadaptation-enabled machine-learning techniques, and show characteristics of a baseline RL model using these metrics.

show abstract

“…Another approach is to regularize the model with respect to past task knowledge while training the new task. This can either be done by regularizing the model in the weight space (i.e., penalize changes to model parameters) [1,13,30,55,63] or the prediction space (i.e., penalize changes to model predictions) [7,23,33,36,52]. Prediction space regularization (accomplished using knowledge distillation) has been found to perform better than model regularization based methods for class-incremental learning [35,57].…”

Section: Background and Related Workmentioning

confidence: 99%

Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning

Smith¹,

Hsu²,

Balloch³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Modern computer vision applications suffer from catastrophic forgetting when incrementally learning new concepts over time. The most successful approaches to alleviate this forgetting require extensive replay of previously seen data, which is problematic when memory constraints or data legality concerns exist. In this work, we consider the high-impact problem of Data-Free Class-Incremental Learning (DFCIL), where an incremental learning agent must learn new concepts over time without storing generators or training data from past tasks. One approach for DFCIL is to replay synthetic images produced by inverting a frozen copy of the learner's classification model, but we show this approach fails for common class-incremental benchmarks when using standard distillation strategies. We diagnose the cause of this failure and propose a novel incremental distillation strategy for DFCIL, contributing a modified cross-entropy training and importance-weighted feature distillation, and show that our method results in up to a 25.1% increase in final task accuracy (absolute difference) compared to SOTA DFCIL methods for common class-incremental benchmarks. Our method even outperforms several standard replay based methods which store a coreset of images.

show abstract

Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffer

Cited by 8 publications

References 38 publications

Continual Sequence Generation with Adaptive Compositional Modules

Continual Sequence Generation with Adaptive Compositional Modules

NovGrid: A Flexible Grid World for Evaluating Agent Response to Novelty

Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning

Contact Info

Product

Resources

About