A Natural Language Corpus of Common Grounding under Continuous and Partially-Observable Context

Abstract

Common grounding is the process of creating, repairing and updating mutual understandings, which is a critical aspect of sophisticated human communication. However, traditional dialogue systems have limited capability of establishing common ground, and we also lack task formulations which introduce natural difficulty in terms of common grounding while enabling easy evaluation and analysis of complex models. In this paper, we propose a minimal dialogue task which requires advanced skills of common grounding under continuous and partially-observable context. Based on this task formulation, we collected a largescale dataset of 6,760 dialogues which fulfills essential requirements of natural language corpora. Our analysis of the dataset revealed important phenomena related to common grounding that need to be considered. Finally, we evaluate and analyze baseline neural models on a simple subtask that requires recognition of the created common ground. We show that simple baseline models perform decently but leave room for further improvement. Overall, we show that our proposed task will be a fundamental testbed where we can train, evaluate, and analyze dialogue system's ability for sophisticated common grounding.

A Linguistic Analysis of Visually Grounded Dialogues Based on Spatial Expressions

“…Our work extends OneCommon Corpus originally proposed in Udagawa and Aizawa (2019). In this task, two players A and B are given slightly different, overlapping perspectives of a 2-dimensional grid with 7 entities in each view (Figure 1, left).…”

“…This typically involves two types of advanced grounding: symbol grounding (Harnad, 1990), which bridges symbolic natural language and continuous visual perception, and common grounding (Clark, 1996), which refers to the process of developing mutual understandings through successive dialogues. As noted in Monroe et al (2017); Udagawa and Aizawa (2019), the continuous nature of visual context introduces challenging symbol grounding of nuanced and pragmatic expressions. Some further incorporate par-tial observability where the agents do not share the same context, which introduces complex misunderstandings that need to be resolved through advanced common grounding (Udagawa and Aizawa, 2019;Haber et al, 2019).…”

“…As noted in Monroe et al (2017); Udagawa and Aizawa (2019), the continuous nature of visual context introduces challenging symbol grounding of nuanced and pragmatic expressions. Some further incorporate par-tial observability where the agents do not share the same context, which introduces complex misunderstandings that need to be resolved through advanced common grounding (Udagawa and Aizawa, 2019;Haber et al, 2019).…”

Language-Conditioned Feature Pyramids for Visual Selection Tasks

“…Much effort has been taken to improve methods of creating visual common ground between machines, which have limited means of expression and knowledge about the real world, and humans, from the perspectives of both referring expression comprehension and generation (Moratz et al, 2002;Tenbrink and Moratz, 2003;Funakoshi et al, 2004Funakoshi et al, , 2005Funakoshi et al, , 2006. Even now, researchers are exploring possible methods of designing more realistic scenarios for applications, such as in visual dialogue games (De Vries et al, 2017;Haber et al, 2019;Udagawa and Aizawa, 2019).…”

“…For this reason, we focus on the OneCommon corpus (Udagawa and Aizawa, 2019), a recently proposed corpus on a visual dialogue game using composite images of simple figures. It captures various expressions based on positional relationships, such as group-based expressions, as shown in Figure 2.…”