LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5

Abstract

Existing approaches to lifelong language learning rely on plenty of labeled data for learning a new task, which is hard to obtain in most real scenarios. Considering that humans can continually learn new tasks from a handful of examples, we expect the models also to be able to generalize well on new few-shot tasks without forgetting the previous ones. In this work, we define this more challenging yet practical problem as Lifelong Few-shot Language Learning (LFLL) and propose a unified framework for it based on prompt tuning of T5. Our framework called LFPT5 takes full advantage of PT's strong few-shot learning ability, and simultaneously trains the model as a task solver and a data generator. Before learning a new domain of the same task type, LFPT5 generates pseudo (labeled) samples of previously learned domains, and later gets trained on those samples to alleviate forgetting of previous knowledge as it learns the new domain. In addition, a KL divergence loss is minimized to achieve label consistency between the previous and the current model. While adapting to a new task type, LFPT5 includes and tunes additional prompt embeddings for the new task. With extensive experiments, we demonstrate that LFPT5 can be applied to various different types of tasks and significantly outperform previous methods in different LFLL settings.

Progressive Prompts: Continual Learning for Language Models

“…We run extensive experiments on standard CL benchmarks for text classification, and show that Progressive Prompts outperforms state-of-the-art approaches on both BERT and T5 architectures (Devlin et al, 2018;Raffel et al, 2020). We show over 20% improvement over the current SOTA for T5 model (Qin & Joty, 2021). Furthermore, we run experiments on a more challenging CL setup with longer task sequences, and show that our method outperforms prior approaches for T5 and BERT architectures.…”

“…The training objective for task T k (k ∈ {1...m}) is to find prompt parameters θ P k that minimize the negative log probability of training examples under our progressive prompt and the frozen base model: Large number of tasks While previous CL approaches mostly focused on relatively short task sequences of 3-5 tasks (Huang et al, 2021;de Masson D'Autume et al, 2019;Qin & Joty, 2021), a more realistic CL scenario would be long sequences encompassing more tasks. Hence, we create a benchmark of 15 text classification tasks to evaluate the performance of Progressive Prompts along with widely adopted CL approaches.…”

“…• PromptTuning Qin & Joty, 2021): train a shared soft prompt sequentially on all tasks, while keeping the original model parameters frozen.…”

“…• LFPT5 (Qin & Joty, 2021): continuously train a soft prompt that simultaneously learns to solve the tasks and generate training samples, which are subsequently used in experience replay. Current SOTA on CL benchmark with T5.…”

A Roadmap for Big Model

“…Along with continual learning, the parameters of big models are also continually updated, which means that those prompts tuned based on historic parameters may fail based on the latest version of big models. Therefore, it is meaningful to ensure that the prompts tuned on historic parameters can continue to work on new models, which has been demonstrated in the paper [572].…”