Grounding ‘Grounding’ in NLP

Abstract

The NLP community has seen substantial recent interest in grounding to facilitate interaction between language technologies and the world. However, as a community, we use the term broadly to reference any linking of text to data or non-textual modality. In contrast, Cognitive Science more formally defines "grounding" as the process of establishing what mutual information is required for successful communication between two interlocutorsa definition which might implicitly capture the NLP usage but differs in intent and scope.We investigate the gap between these definitions and seek answers to the following questions: (1) What aspects of grounding are missing from NLP tasks? Here we present the dimensions of coordination, purviews and constraints.(2) How is the term "grounding" used in the current research? We study the trends in datasets, domains, and tasks introduced in recent NLP conferences. And finally, (3) How to advance our current definition to bridge the gap with Cognitive Science? We present ways to both create new tasks or repurpose existing ones to make advancements towards achieving a more complete sense of grounding.

Conversational AI for multi-agent communication in Natural Language

“…Decisions can be made at the level of dialogue acts [24,34,52], words [18], gestures, or robot actions, or combinations of these [28]. We have also used Deep RL for learning communication policies in strategic non-cooperative games 8 such as negotiations in the game Settlers of Catan [10].…”

“…A central reason for the lack of work on conversational collaborative interaction in multi-agent systems is that current datasets do not contain semantic coordination phenomena of the type that will allow such skills to be learned. This is because (as argued by [8,25,35,41,54]) these datasets do not focus on agents with different goals and/or knowledge of the task that needs to be coordinated. The few exceptions, such as Cups [41] and MeetUp [25], have only small volumes of data, and almost no datasets go beyond pairs of agents.…”

Describe me an Aucklet: Generating Grounded Perceptual Category Descriptions

“…Language is one of the means we use to achieve our goals when acting in the real world (Chandu et al, 2021). We link forms (symbols) with meaning rooted in other modalities such as perception, sensormotorics, and sounds among others.…”

She adapts to her student: An expert pragmatic speaker tailoring her referring expressions to the Layman listener

“…In the development of conversational agents, this core aspect was on the agenda of the pre-neural era (e.g., Janarthanam and Lemon, 2010), but it has been put aside for several years since the main challenge has been to obtain conversational neural models able to produce utterances and dialogues that are human-like at least at the surface level. Now that this milestone has been achieved, the importance of paying attention to the dynamic and adaptive interactive aspect of language has been advocated in several position papers (Bisk et al, 2020;Benotti and Blackburn, 2021;Chandu et al, 2021). Inspired by the seminal work on multi-agent settings carried out within the language emergence research line (Lazaridou et al, 2017(Lazaridou et al, , 2020, the community has also been challenged to consider multiagent interactions in which a speaker has to interact and adapt to a population of listeners with different visual (Corona et al, 2019), or different linguistic (Zhu et al, 2021) abilities.…”