Reflective Decoding: Beyond Unidirectional Generation with Off-the-Shelf Language Models

Abstract

Publicly available, large pretrained Language Models (LMs) generate text with remarkable quality, but only sequentially from left to right. As a result, they are not immediately applicable to generation tasks that break the unidirectional assumption, such as paraphrasing or text-infilling, necessitating task-specific supervision.In this paper, we present REFLECTIVE DE-CODING, a novel unsupervised algorithm that allows for direct application of unidirectional LMs to non-sequential tasks. Our 2-step approach requires no supervision or even parallel corpora, only two off-the-shelf pretrained LMs in opposite directions: forward and backward. First, in the contextualization step, we use LMs to generate ensembles of past and future contexts which collectively capture the input (e.g. the source sentence for paraphrasing). Second, in the reflection step, we condition on these "context ensembles", generating outputs that are compatible with them. Comprehensive empirical results 1 demonstrate that REFLECTIVE DECODING outperforms strong unsupervised baselines on both paraphrasing and abductive text infilling, significantly narrowing the gap between unsupervised and supervised methods. REFLECTIVE DECODING surpasses multiple supervised baselines on various metrics including human evaluation.

InCoder: A Generative Model for Code Infilling and Synthesis

“…Unfortunately, standard left-to-right language models cannot directly infill text, and popular masked language models are mainly trained to infill very short spans [17,22,52,54]. Recent work addresses this by changing model architectures, inference procedures, and training objectives [2,59,67,3]. Most related to our approach is the work of Donahue et al [23] and CM3 [2], who train left-to-right language models to fill in masked token segments of varying lengths; and the work of Alon et al [7], who train an infilling-capable, AST-structured generative model of code on a smaller scale.…”

Knowledge Infused Decoding

“…In Figure 2 (c), we quantify the exposure bias problem through human judgements. We first sample 200 ELI5 test set questions and generate answers of various lengths {80, 100, ..., 260} (260 is the average sequence length in training set) with beam search, sampling, reflective (West et al, 2021), and KID. We then ask humans to rate these generations with 7-point Likert scoring (Joshi et al, 2015) how likely the generated text is a natural sentence.…”

“…None of these decoding algorithms consider integrating knowledge in the generation process. Reflective decoding (West et al, 2021) and DeLorean (Qin et al, 2020) are two recent decoding algorithms that focus on abductive commonsense reasoning. Reflective decoding in particular has the potential to be extended to other knowledge-intensive tasks.…”

“…Besides RAG(Lewis et al, 2020b), we consider several competitive prior methods incorporating knowledge including a) GPT2 + Knowledge(Guan et al, 2020), which post-trains GPT2 on triplets-converted augmentation data (e.g., ⟨helium, is, gas⟩ → "helium is gas. "); b) GPT2 + COMeT Embeddings, which fuses knowledge-aware embeddings (e.g., from CoMET(Bosselut et al, 2019); c) FiD-T5 (Izacard & Grave, 2021b;a), which concatenates retrieved grounding documents with context as new input; d) QA-GNN(Yasunaga et al, 2021), which traverses a graph neural network; e) Reflective decoding(West et al, 2021), which relies on forward and backward LMs to encode bi-directional context for generation.As shown in Table2(columns with 100% training data), KID outperforms all other methods in ELI5 (with 5.2 ROUGE-L points improvements over the second-best method (RAG)), and achieves competitive results requiring neither specific model architecture nor additional training to infuse knowledge.…”