Towards a model of face-to-face grounding

Abstract

We investigate the verbal and nonverbal means for grounding, and propose a design for embodied conversational agents that relies on both kinds of signals to establish common ground in human-computer interaction. We analyzed eye gaze, head nods and attentional focus in the context of a direction-giving task. The distribution of nonverbal behaviors differed depending on the type of dialogue move being grounded, and the overall pattern reflected a monitoring of lack of negative feedback. Based on these results, we present an ECA that uses verbal and nonverbal grounding acts to update dialogue state.

Pursuing and demonstrating understanding in dialogue

“…The system interpreted these cues as grounding actions using an incremental common ground model, and varied its generation strategies accordingly. Nakano et al [2003] found that their system elicited from its interlocutors many of the same qualitative dynamics they found in human-human conversations.…”

“…The system detects and adapts to the nonverbal grounding cues that followers spontaneously provide in human-human conversations. For example, Nakano et al [2003] found that when listeners could follow instructions, they would nod and continue to direct their attention to the map, but when something was unclear, listeners would gaze up toward the direction-giver and wait attentively for clarification. The system interpreted these cues as grounding actions using an incremental common ground model, and varied its generation strategies accordingly.…”

“…By using complex communicative actions that pair spoken utterances with actions in other modalities, embodied conversational agents can communicate their understanding in flexible and natural new ways. For example, Nakano et al [2003] describe an embodied conversational agent that gives directions with reference to a tabletop map. The system detects and adapts to the nonverbal grounding cues that followers spontaneously provide in human-human conversations.…”

Perspectives on Dialogue: Introduction to this Special Issue

“…This requires further research into the integrated use in dialogue of verbal and non-verbal means (see, e.g., Nakano et al, 2003 ). From the point of view of applications, this is an area which has recently gained interest in the context of the construction of Embodied Conversational Agents that is, computer-animated characters that can engage in a dialogue with a user or other computer-animated characters (e.g., Cassell et al, 2000;Prendinger and Ishizuka, 2004).…”

Attentive Presentation Agents

“…They can perform several gestures like greeting, counting with fingers, deictic and beat gestures, and facial expressions (happy, sad, surprised). Speech output is generated using Loquendo Text-to-Speech [9], and lips are adequately animated.…”