Multi-Relational Question Answering from Narratives: Machine Reading and Reasoning in Simulated Worlds

Abstract

Question Answering (QA), as a research field, has primarily focused on either knowledge bases (KBs) or free text as a source of knowledge. These two sources have historically shaped the kinds of questions that are asked over these sources, and the methods developed to answer them. In this work, we look towards a practical use-case of QA over user-instructed knowledge that uniquely combines elements of both structured QA over knowledge bases, and unstructured QA over narrative, introducing the task of multirelational QA over personal narrative. As a first step towards this goal, we make three key contributions: (i) we generate and release TEXTWORLDSQA, a set of five diverse datasets, where each dataset contains dynamic narrative that describes entities and relations in a simulated world, paired with variably compositional questions over that knowledge, (ii) we perform a thorough evaluation and analysis of several state-of-the-art QA models and their variants at this task, and (iii) we release a lightweight Python-based framework we call TEXTWORLDS for easily generating arbitrary additional worlds and narrative, with the goal of allowing the community to create and share a growing collection of diverse worlds as a test-bed for this task. . 2014. Learning phrase representations using rnn encoder-decoder for statistical machine translation. In EMNLP.

CoQA: A Conversational Question Answering Challenge

“…We use the Document Reader (DrQA) model of Chen et al (2017), which has demonstrated strong performance on multiple datasets (Rajpurkar et al, 2016;Labutov et al, 2018). Because DrQA requires text spans as answers during training, we select the span that has the highest lexical overlap (F1 score) with the original answer as the gold answer.…”

A Vietnamese Dataset for Evaluating Machine Reading Comprehension

“…• DrQA: Chen et al (2017) introduced a simple but effective neural network-based model for the MRC task. DrQA Reader achieved good performance on multiple MRC datasets (Rajpurkar et al, 2016;Reddy et al, 2019;Labutov et al, 2018). Thus, we re-implement this method into our dataset as the first baseline models to compare future models.…”

Conversational Machine Reading Comprehension for Vietnamese Healthcare Texts

“…For the machine reading comprehension model, the Document Reader (DrQA) introduced by Chen et al [1] is a powerful model on various of machine reading comprehension corpora such as: SQuAD [11], TextWorldsQA [8], and UIT-ViSQuAD [10]. The DrQA model consists of two modules: Document Retriever and Document Reader.…”