Interactive Language Learning by Question Answering

Abstract

Humans observe and interact with the world to acquire knowledge. However, most existing machine reading comprehension (MRC) tasks miss the interactive, information-seeking component of comprehension. Such tasks present models with static documents that contain all necessary information, usually concentrated in a single short substring. Thus, models can achieve strong performance through simple word-and phrase-based pattern matching. We address this problem by formulating a novel text-based question answering task: Question Answering with Interactive Text (QAit). 1 In QAit, an agent must interact with a partially observable text-based environment to gather information required to answer questions. QAit poses questions about the existence, location, and attributes of objects found in the environment. The data is built using a text-based game generator that defines the underlying dynamics of interaction with the environment. We propose and evaluate a set of baseline models for the QAit task that includes deep reinforcement learning agents. Experiments show that the task presents a major challenge for machine reading systems, while humans solve it with relative ease.

Visually Grounded Continual Learning of Compositional Phrases

“…Capturing compositionality in language has been a long challenge (Fodor et al, 1988) for neural networks. Recent works explore the problem with compositional generalization on synthetic instruction following (Lake and Baroni, 2017), text-based games (Yuan et al, 2019), visual question answering (Bahdanau et al, 2019), and visually grounded masked word prediction (Surís et al, 2019). In particular, study a closely related task of continual learning of sequence prediction for synthetic instruction following.…”

Interactive Machine Comprehension with Information Seeking Agents

“…iMRC is distinct from this body of literature in that it does not depend on extra meta information to build tree structures, it is partially-observable, and its action space is as large as 200,000 (much larger than, e.g., the 5 query templates in (Narasimhan et al, 2016) and tree search in (Geva and Berant, 2018)). Our work is also inspired directly by QAit (Yuan et al, 2019), a set of interactive question answering tasks developed on text-based games. However, QAit is based on synthetic and templated language which might not require strong language understanding components.…”

“…As a baseline agent, we adopt QA-DQN (Yuan et al, 2019), we modify it to enable extractive QUERY generation and question answering.…”

“…In this section, we only describe the difference between the model our baseline agent uses and the original QA-DQN. We refer readers to (Yuan et al, 2019) for detailed information.…”

“…Because the question answerer in our agent is a pointing model, its performance relies heavily on whether the agent can find and stop at the sentence that contains the answer. In the same spirit as (Yuan et al, 2019), we also design a heuristic reward to guide agents to learn this behavior.…”