Read before Generate! Faithful Long Form Question Answering with Machine Reading

Abstract

Long-form question answering (LFQA) aims to generate a paragraph-length answer for a given question. While current work on LFQA using large pre-trained model for generation are effective at producing fluent and somewhat relevant content, one primary challenge lies in how to generate a faithful answer that has less hallucinated content. We propose a new end-to-end framework that jointly models answer generation and machine reading. The key idea is to augment the generation model with fine-grained, answer-related salient information which can be viewed as an emphasis on faithful facts. State-of-the-art results on two LFQA datasets, ELI5 and MS MARCO, demonstrate the effectiveness of our method, in comparison with strong baselines on automatic and human evaluation metrics. A detailed analysis further proves the competency of our methods in generating fluent, relevant, and more faithful answers.

How Do We Answer Complex Questions: Discourse Structure of Long-form Answers

“…This shares motivation with a line of work studying query-focused summarization (Xu and Lapata, 2020). Concurrent to our work, Su et al (2022) studies improving faithfulness of long-form answer through predicting and focusing on salient information in retrieved evidence document. Lastly, our work build up on three datasets containing longform answers (Kwiatkowski et al, 2019;Fan et al, 2019;Nakano et al, 2021) and extends the analysis of long-form answers from earlier studies (Krishna et al, 2021).…”

Proposed Framework for Human-like Language Processing of ChatGPT in Academic Writing

“…Most developed language models produced nonfactual information [23], [24], [25] and [26] similar to other LLMs [27], [28], [29] and [30], the ChatGPT tool hallucinated some facts. For example, in Figure 6 below, when the researchers asked the tool about the flag of Yemen, a part of the generated response was correct, while another part of the information was revealed to be incorrect when verifying with the source.…”

Improving Query-Focused Meeting Summarization with Query-Relevant Knowledge

“…We choose BART-large (Lewis et al, 2020), a transformer-based (Vaswani et al, 2017) generative pre-trained language model, as our backbone model for the generator because of its remarkable performance on text summarization benchmarks. Following the idea of Fusion-in-Decoder (FiD) and its applications in generation tasks (Izacard and Grave, 2021;Su et al, 2022;Vig et al, 2022), we employ FiD-BART for encoding multiple segments independently in the encoder and fuse information from all segments in the decoder jointly through the encoder-decoder attention.…”