Augmenting Transformers with KNN-Based Composite Memory for Dialog

Fan, Angela; Gardent, Claire; Braud, Chloé; Bordes, Antoine

doi:10.1162/tacl_a_00356

Cited by 26 publications

(28 citation statements)

References 21 publications

(39 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Retrieval-augmented models. Retrievalaugmented models are now widely adopted in open-domain question answering (Chen et al, 2017;de Masson d'Autume et al, 2019;Izacard and Grave, 2021), dialogue (Dinan et al, 2019;Fan et al, 2021;Thulke et al, 2021) and machine translation (Bapna and Firat, 2019;Khandelwal et al, 2020a). We focus on retrieval augmentation for language modelling (Merity et al, 2017;Grave et al, 2016;Khandelwal et al, 2020b;Yogatama et al, 2021).…”

Section: Knowledge-enhancedmentioning

confidence: 99%

Relational Memory Augmented Language Models

Liu¹,

Yogatama²,

Blunsom³

2022

Preprint

View full text Add to dashboard Cite

We present a memory-augmented approach to condition an autoregressive language model on a knowledge graph. We represent the graph as a collection of relation triples and retrieve relevant relations for a given context to improve text generation. Experiments on WikiText-103, WMT19, and en-wik8 English datasets demonstrate that our approach produces a better language model in terms of perplexity and bits per character. We also show that relational memory improves coherence, is complementary to token-based memory, and enables causal interventions. Our model provides a simple yet effective way to combine an autoregressive language model and a knowledge graph for more coherent and logical generation.

show abstract

Section: Knowledge-enhancedmentioning

confidence: 99%

Relational Memory Augmented Language Models

Liu¹,

Yogatama²,

Blunsom³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Recent efforts in NLP have shown the effectiveness of relying on an explicit set of nearest neighbors to be effective for language modelling , question answering (Kassner and Schütze, 2020) and knowledge-grounded dialog (Fan et al, 2020). However, these approaches condition on examples only during inference or in a non end-to-end manner.…”

Section: Example-driven Trainingmentioning

confidence: 99%

Example-Driven Intent Prediction with Observers

Mehri

Eric

Hakkani‐Tür

2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

A key challenge of dialog systems research is to effectively and efficiently adapt to new domains. A scalable paradigm for adaptation necessitates the development of generalizable models that perform well in few-shot settings. In this paper, we focus on the intent classification problem which aims to identify user intents given utterances addressed to the dialog system. We propose two approaches for improving the generalizability of utterance classification models: (1) observers and (2) example-driven training. Prior work has shown that BERT-like models tend to attribute a significant amount of attention to the [CLS] token, which we hypothesize results in diluted representations. Observers are tokens that are not attended to, and are an alternative to the [CLS] token as a semantic representation of utterances. Example-driven training learns to classify utterances by comparing to examples, thereby using the underlying encoder as a sentence similarity model. These methods are complementary; improving the representation through observers allows the example-driven model to better measure sentence similarities. When combined, the proposed methods attain state-of-the-art results on three intent prediction datasets (BANKING77, CLINC150, HWU64) in both the full data and few-shot (10 examples per intent) settings. Furthermore, we demonstrate that the proposed approach can transfer to new intents and across datasets without any additional training.

show abstract

“…Marino et al (2019) introduced OK-VQA, a novel VQA dataset that requires the use of an external KS. Fan et al (2020) applied a KS to multi-modal dialogue. In our work, we focus on a more naturally aligned KS, in the form of images and captions, which better reflects the data generated in newspapers and social media.…”

Section: Related Workmentioning

confidence: 99%

Cross-Modal Retrieval Augmentation for Multi-Modal Classification

Gur¹,

Neverova²,

Stauffer³

et al. 2021

Findings of the Association for Computational Linguistics: EMNLP 2021

View full text Add to dashboard Cite

Recent advances in using retrieval components over external knowledge sources have shown impressive results for a variety of downstream tasks in natural language processing. Here, we explore the use of unstructured external knowledge sources of images and their corresponding captions for improving visual question answering (VQA). First, we train a novel alignment model for embedding images and captions in the same space, which achieves substantial improvements in performance on image-caption retrieval w.r.t. similar methods. Second, we show that retrieval-augmented multi-modal transformers using the trained alignment model improve results on VQA over strong baselines. We further conduct extensive experiments to establish the promise of this approach, and examine novel applications for inference time such as hot-swapping indices.

show abstract

Augmenting Transformers with KNN-Based Composite Memory for Dialog

Cited by 26 publications

References 21 publications

Relational Memory Augmented Language Models

Relational Memory Augmented Language Models

Example-Driven Intent Prediction with Observers

Cross-Modal Retrieval Augmentation for Multi-Modal Classification

Contact Info

Product

Resources

About