Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering

Chen, Wenhu; Verga, Pat; Jong, Michiel de; Wieting, John; Cohen, William W.

doi:10.48550/arxiv.2204.04581

Cited by 4 publications

(6 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…, (q k , a k ), concatenating them with the original question q , and then using a Transformer to fuse this information and generate a final combined answer a . The model we used, called QAMAT (for Question-Answer Memory Augmented Transformer), is described in detail elsewhere (Chen et al 2022).…”

Section: Methodsmentioning

confidence: 99%

QA Is the New KR: Question-Answer Pairs as Knowledge Bases

Cohen

Chen

Jong

et al. 2023

AAAI

View full text Add to dashboard Cite

We propose a new knowledge representation (KR) based on knowledge bases (KBs) derived from text, based on question generation and entity linking. We argue that the proposed type of KB has many of the key advantages of a traditional symbolic KB: in particular, it consists of small modular components, which can be combined compositionally to answer complex queries, including relational queries and queries involving ``multi-hop'' inferences. However, unlike a traditional KB, this information store is well-aligned with common user information needs. We present one such KB, called a QEDB, and give qualitative evidence that the atomic components are high-quality and meaningful, and that atomic components can be combined in ways similar to the triples in a symbolic KB. We also show experimentally that questions reflective of typical user questions are more easily answered with a QEDB than a symbolic KB.

show abstract

Section: Methodsmentioning

confidence: 99%

QA Is the New KR: Question-Answer Pairs as Knowledge Bases

Cohen

Chen

Jong

et al. 2023

AAAI

View full text Add to dashboard Cite

show abstract

“…Question Generation Question generation has been successfully applied to various purposes, including augmenting question answering systems (Duan et al, 2017;Lewis et al, 2021), capturing implicit information written about text (Pyatkin et al, 2021), and building soft knowledge bases (Chen et al, 2022). In this work, we apply question generation to the task of discriminating edited sentences.…”

Section: Modelsmentioning

confidence: 99%

“…Question generation (QG) is a new NLP task that consists of generating a question that a provided document answers. There are various successful applications of this approach, including augmenting datasets to train question answering systems (Duan et al, 2017;Lewis et al, 2021), capturing implicit information written about text (Pyatkin et al, 2021), and building soft knowledge bases (Chen et al, 2022). Previous work in QG treated the underlying passages as static (Lewis et al, 2021), while real life documents are constantly updated (Dhingra et al, 2022).…”

Section: Introductionmentioning

confidence: 99%

DiffQG: Generating Questions to Summarize Factual Changes

Cole,

Jain,

Eisenschlos

et al. 2023

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics

View full text Add to dashboard Cite

Identifying the difference between two versions of the same article is useful to update knowledge bases and to understand how articles evolve. Paired texts occur naturally in diverse situations: reporters write similar news stories and maintainers of authoritative websites must keep their information up to date. We propose representing factual changes between paired documents as question-answer pairs, where the answer to the same question differs between two versions. We find that question-answer pairs can flexibly and concisely capture the updated contents. Provided with paired documents, annotators identify questions that are answered by one passage but answered differently or cannot be answered by the other. We release DIFFQG which consists of 759 QA pairs and 1153 examples of paired passages with no factual change. These questions are intended to be both unambiguous and information-seeking and involve complex edits, pushing beyond the capabilities of current question generation and factual change detection systems. Our dataset summarizes the changes between two versions of the document as questions and answers, studying automatic update summarization in a novel way.

show abstract

“…To avoid the excessive computation cost of backpropagation over the massive external memory, we adopt an in-batch memory M B , dynamically constructed from the input examples in a batch. The small in-batch memory enables MuRAG to continuously update the memory encoder efficiently similar to TOME (de Jong et al, 2022) and QAMAT (Chen et al, 2022).…”

Section: Pre-trainingmentioning

confidence: 99%

MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text

Chen¹,

Hu²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

While language Models store a massive amount of world knowledge implicitly in their parameters, even very large models often fail to encode information about rare entities and events, while incurring huge computational costs. Recently, retrieval-augmented models, such as REALM, RAG, and RETRO, have incorporated world knowledge into language generation by leveraging an external nonparametric index and have demonstrated impressive performance with constrained model sizes. However, these methods are restricted to retrieving only textual knowledge, neglecting the ubiquitous amount of knowledge in other modalities like images -much of which contains information not covered by any text. To address this limitation, we propose the first Multimodal Retrieval-Augmented Transformer (MuRAG), which accesses an external non-parametric multimodal memory to augment language generation. MuRAG is pretrained with a mixture of large-scale imagetext and text-only corpora using a joint contrastive and generative loss. We perform experiments on two different datasets that require retrieving and reasoning over both images and text to answer a given query: We-bQA, and MultimodalQA. Our results show that MuRAG achieves state-of-the-art accuracy, outperforming existing models by 10-20% absolute on both datasets and under both distractor and full-wiki settings.

show abstract

Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering

Cited by 4 publications

References 20 publications

QA Is the New KR: Question-Answer Pairs as Knowledge Bases

QA Is the New KR: Question-Answer Pairs as Knowledge Bases

DiffQG: Generating Questions to Summarize Factual Changes

MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text

Contact Info

Product

Resources

About