We investigate the verbal and nonverbal means for grounding, and propose a design for embodied conversational agents that relies on both kinds of signals to establish common ground in human-computer interaction. We analyzed eye gaze, head nods and attentional focus in the context of a direction-giving task. The distribution of nonverbal behaviors differed depending on the type of dialogue move being grounded, and the overall pattern reflected a monitoring of lack of negative feedback. Based on these results, we present an ECA that uses verbal and nonverbal grounding acts to update dialogue state.

People cannot type as fast as they think, especially when faced with the constraints of mobile devices. There have been numerous approaches to solving this problem, including research in augmented input devices and predictive typing aids. We propose an alternative approach to predictive text entry based on commonsense reasoning. Using OMCSNet, a large-scale semantic network that aggregates and normalizes the contributions made to Open Mind Common Sense (OMCS), our system is able to show significant success in predicting words based on their first few letters. We evaluate this commonsense approach against traditional statistical methods, demonstrating comparable performance, and suggest that combining commonsense and statistical approaches could achieve superior performance. Mobile device implementations of the commonsense predictive typing aid demonstrate that such a system could be applied to just about any computing environment.

No abstract

No abstract

In this paper, we describe an interface that demonstrates spatial intelligence. This interface, an embodied conversational kiosk, builds on research in embodied conversational agents (ECAs) and on information displays in mixed reality and kiosk format. ECAs leverage people's abilities to coordinate information displayed in multiple modalities, particularly information conveyed in speech and gesture. Mixed reality depends on users' interactions with everyday objects that are enhanced with computational overlays. We describe an implementation, MACK (Media lab Autonomous Conversational Kiosk), an ECA who can answer questions about and give directions to the MIT Media Lab's various research groups, projects and people. MACK uses a combination of speech, gesture, and indications on a normal paper map that users place on a table between themselves and MACK. Research issues involve users' differential attention to hand gestures, speech and the map, and how reference using these modalities can be fused in input and generation.