The Power of Scale for Parameter-Efficient Prompt Tuning

Abstract

In this work, we explore "prompt tuning," a simple yet effective mechanism for learning "soft prompts" to condition frozen language models to perform specific downstream tasks. Unlike the discrete text prompts used by GPT-3, soft prompts are learned through backpropagation and can be tuned to incorporate signals from any number of labeled examples. Our end-to-end learned approach outperforms GPT-3's few-shot learning by a large margin. More remarkably, through ablations on model size using T5, we show that prompt tuning becomes more competitive with scale: as models exceed billions of parameters, our method "closes the gap" and matches the strong performance of model tuning (where all model weights are tuned). This finding is especially relevant because large models are costly to share and serve and the ability to reuse one frozen model for multiple downstream tasks can ease this burden. Our method can be seen as a simplification of the recently proposed "prefix tuning" of Li and Liang (2021) and we provide a comparison to this and other similar approaches. Finally, we show that conditioning a frozen model with soft prompts confers benefits in robustness to domain transfer and enables efficient "prompt ensembling." We release code and model checkpoints to reproduce our experiments. 1

PromDA: Prompt-based Data Augmentation for Low-Resource NLU Tasks

“…However, the selection of discrete prompts is still an independent process and cannot be optimized together with the downstream tasks in an end-to-end manner. To solve this issue, (Lester et al, 2021;Li and Liang, 2021) propose to use soft prompts, which are sets of trainable vectors, in the frozen pretrained language models. Unlike the hard prompts, these vectors do not correspond to any real words.…”

“…However, finding suitable discrete task introductions cannot be easily optimized in an end-to-end fashion and requires extra human effort. In this paper, inspired by the recent work (Lester et al, 2021;Li and Liang, 2021), we replace the task introductions with Soft Prompt (i.e., a sequence of continuous and trainable vectors). During training, we only update the parameters of this Soft Prompt and fix all PLMs parameters.…”

“…Unlike Lester et al (2021) which only prepends Soft Prompt at the input layer, inspired by Adaptor (Houlsby et al, 2019) which adds trainable Multi-layer Perceptron (MLP) at each transformer layer, we prepend a sequence of trainable vectors at each transformer layer. We denote P j = {p j 1 , • • • , p j k } as the Soft Prompt at the j th layer.…”

“…As a result, the generated synthetic data could be very similar to the original training instances and cannot provide new training signals to the NLU models. Recently, several works (Lester et al, 2021;Li and Liang, 2021) propose prompt tuning, which only back-propagates the error to Soft Prompts (i.e., a sequence of continuous vectors prepended to the input of PLMs) instead of the entire model. They show that prompt tuning is sufficient to be competitive with full model tuning while significantly reducing the amount of parameters to be tuned.…”

Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models

“…Specifically, we follow the definition of toxicity from Perspective API as well as the inspiration by the recent work from Prompt Engineering (Liu et al, 2021b;Shin et al, 2020;Li & Liang, 2021;Lester et al, 2021;Zhao et al, 2021;Schick & Schütze, 2020a; that repeating the prompts and prompting LMs in the format of Question Answering, and design the prompts below to study the generation and understanding power of the LMs, 1. Negative Prompt (for once).…”