Open-Domain Question Answering Goes Conversational via Question Rewriting

Abstract

We introduce a new dataset for Question Rewriting in Conversational Context (QReCC), which contains 14K conversations with 81K question-answer pairs. The task in QReCC is to find answers to conversational questions within a collection of 10M web pages (split into 54M passages). Answers to questions in the same conversation may be distributed across several web pages. QReCC provides annotations that allow us to train and evaluate individual subtasks of question rewriting, passage retrieval and reading comprehension required for the end-to-end conversational question answering (QA) task. We report the effectiveness of a strong baseline approach that combines the state-of-the-art model for question rewriting, and competitive models for open-domain QA. Our results set the first baseline for the QReCC dataset with F1 of 19.07, compared to the human upper bound of 74.47, indicating the difficulty of the setup and a large room for improvement.

A Comparison of Question Rewriting Methods for Conversational Passage Retrieval

“…Following previous work, we perform both intrinsic and extrinsic evaluation [2,8]. In intrinsic evaluation, we compare rewrites produced by QR methods with manual rewrites produced by human annotators using ROUGE-1 Precision (P), Recall (R) and F-measure (F) [2].…”

“…Following previous work, we perform both intrinsic and extrinsic evaluation [2,8]. In intrinsic evaluation, we compare rewrites produced by QR methods with manual rewrites produced by human annotators using ROUGE-1 Precision (P), Recall (R) and F-measure (F) [2]. 4 In extrinsic evaluation, we measure PR performance when using different QR methods using standard ranking metrics: NDCG@3, MRR and Recall@1000.…”

Neural Approaches to Conversational AI

“…Both open-domain QA and ad hoc search datasets are frequently used as additional (weak) supervision data in conversational search. Recently, the QReCC dataset [Anantha et al, 2020] is released as yet another conversational search benchmark. It extends CAsT and QuAC with additional conversations constructed from Natural Questions and additional passages sampled from Common Crawl.…”

From Rewriting to Remembering: Common Ground for Conversational QA Models

“…The approach has a main shortcoming, namely, it introduces a great amount of noise, since not everything in the previous turns is relevant. An alternative approach is Question Rewriting (QR), in which the question is rewritten in a self-contained form based on the previous conversational information (Vakulenko et al, 2021a;Anantha et al, 2020). QR selects only the relevant information in previous turns, thus improving over concatenation.…”