Using Task Descriptions in Lifelong Machine Learning for Improved Performance and Zero-Shot Transfer

Isele, David; Rostami, Mohammad; Eaton, Eric

doi:10.1613/jair.1.11304

Cited by 23 publications

(30 citation statements)

References 21 publications

(28 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To our knowledge, the only work somehow resembling ours is (Rostami et al, 2020), in which task descriptions were incorporated into lifelong learning for zero-shot transfer. We differ in three aspects: (i) they focused on robot controlling problems, (ii) their tasks are from a single domain, and (iii) in addition to the associated instruction, they assumed that each task has a large number of labeled examples.…”

Section: Related Workmentioning

confidence: 99%

ConTinTin: Continual Learning from Task Instructions

Yin¹,

Li²,

Xiong³

2022

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

View full text Add to dashboard Cite

The mainstream machine learning paradigms for NLP often work with two underlying presumptions. First, the target task is predefined and static; a system merely needs to learn to solve it exclusively. Second, the supervision of a task mainly comes from a set of labeled examples. A question arises: how to build a system that can keep learning new tasks from their instructions?This work defines a new learning paradigm ConTinTin (Continual Learning from Task Instructions), in which a system should learn a sequence of new tasks one by one, each task is explained by a piece of textual instruction. The system is required to (i) generate the expected outputs of a new task by learning from its instruction, (ii) transfer the knowledge acquired from upstream tasks to help solve downstream tasks (i.e., forward-transfer), and (iii) retain or even improve the performance on earlier tasks after learning new tasks (i.e., backward-transfer). This new problem is studied on a stream of more than 60 tasks, each equipped with an instruction. Technically, our method InstructionSpeak contains two strategies that make full use of task instructions to improve forward-transfer and backward-transfer: one is to learn from negative outputs, the other is to re-visit instructions of previous tasks. To our knowledge, this is the first time to study ConTinTin in NLP. In addition to the problem formulation and our promising approach, this work also contributes to providing rich analyses for the community to better understand this novel learning problem.

show abstract

Section: Related Workmentioning

confidence: 99%

ConTinTin: Continual Learning from Task Instructions

Yin¹,

Li²,

Xiong³

2022

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

View full text Add to dashboard Cite

show abstract

“…PG-ELLA follows the dictionary-learning mechanics of ELLA, but replaces the supervised models that form ELLA's dictionary by policy factors (Bou Ammar et al, 2014). An extension of this approach supports cross-domain transfer by projecting the dictionary onto domain-specific policy spaces (Bou Ammar et al, 2015), and another extension leverages task descriptors to achieve zero-shot transfer to unseen tasks (Isele et al, 2016;Rostami et al, 2020). Zhao et al ( 2017) followed a similar dictionary-learning formulation for deep networks, replacing all matrix operations with equivalent tensor operations.…”

Section: Lifelong Reinforcement Learningmentioning

confidence: 99%

How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition

A.¹,

Eaton²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

A major goal of artificial intelligence (AI) is to create an agent capable of acquiring a general understanding of the world. Such an agent would require the ability to continually accumulate and build upon its knowledge as it encounters new experiences. Lifelong or continual learning addresses this setting, whereby an agent faces a continual stream of problems and must strive to capture the knowledge necessary for solving each new task it encounters. If the agent is capable of accumulating knowledge in some form of compositional representation, it could then selectively reuse and combine relevant pieces of knowledge to construct novel solutions. Despite the intuitive appeal of this simple idea, the literatures on lifelong learning and compositional learning have proceeded largely separately. In an effort to promote developments that bridge between the two fields, this article surveys their respective research landscapes and discusses existing and future connections between them.

show abstract

“…The idea is to identify important weights that retain knowledge about a task and then consolidate them according to their relative importance for past tasks in the future. Continual learning of sequential tasks can be improved used high-level tasks descriptors to compensate for data scarcity Rostami et al (2020a).…”

Section: Related Workmentioning

confidence: 99%

Cognitively Inspired Learning of Incremental Drifting Concepts

Rostami¹,

Galstyan²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Humans continually expand their learned knowledge to new domains and learn new concepts without any interference with past learned experiences. In contrast, machine learning models perform poorly in a continual learning setting, where input data distribution changes over time. Inspired by the nervous system learning mechanisms, we develop a computational model that enables a deep neural network to learn new concepts and expand its learned knowledge to new domains incrementally in a continual learning setting. We rely on the Parallel Distributed Processing theory to encode abstract concepts in an embedding space in terms of a multimodal distribution. This embedding space is modeled by internal data representations in a hidden network layer. We also leverage the Complementary Learning Systems theory to equip the model with a memory mechanism to overcome catastrophic forgetting through implementing pseudo-rehearsal. Our model can generate pseudo-data points for experience replay and accumulate new experiences to past learned experiences without causing cross-task interference.

show abstract

Using Task Descriptions in Lifelong Machine Learning for Improved Performance and Zero-Shot Transfer

Cited by 23 publications

References 21 publications

ConTinTin: Continual Learning from Task Instructions

ConTinTin: Continual Learning from Task Instructions

How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition

Cognitively Inspired Learning of Incremental Drifting Concepts

Contact Info

Product

Resources

About