Summary. The Behavior Expression Animation Toolkit (BEAT) allows animators to input typed text that they wish to be spoken by an animated human figure, and to obtain as output appropriate and synchronized non-verbal behaviors and synthesized speech in a form that can be sent to a number of different animation systems. The non-verbal behaviors are assigned on the basis of actual linguistic and contextual analysis of the typed text, relying on rules derived from extensive research into human conversational behavior. The toolkit is extensible, so that new rules can be quickly added. It is designed to plug into larger systems that may also assign personality profiles, motion characteristics, scene constraints, or the animation styles of particular animators.

More than another friendly facer Rea knows how to have a conversation with living, breathing human users with a wink, a nod, and a sidelong glance. 70 April 2000/Vot 43, No 4 COMMUNICATIONS OF THE ACM M £0^C OMMUNICATIONS Of THE ACM April ZOOO/Vol, 43 No 4 Hello. How can I help you? Mike. I'm looking to buy a place near MIT Rea nods, indicating that she is following.Rea. I have a house to show you. A picture of a house appears on the screen behind Rea, who blinks and looks at the house and then at Mike. Rea. It's in Somerville.Mike. Tell me about it. Rea looks up and away while she plans what to say.Rea. It's big. Rea makes an expansive gesture with her hands. Mike brings his hands up as if he is about to speak, so Rea does not continue, waiting for him to speak.Mike.Tell me more about it. Rea. Sure thing. It has a nice garden. Rea sketches o curved gesture with her hands, indicating the garden extends along two sides ofthe house.Mike. How far is it? Rea. It is five minutes to the Porter SquareT station. Reo makes it clear it is five minutes on foot from the T station by making a walking gesture with her fingers.Mike. How big is the house? Rea. It has four bedrooms, three bathrooms... Mike interrupts Rea, who stops speaking immediately.Mike. Wait. Tel I me, where is the master bedrooom? Rea. I'm sorry, I didn't catch that. What did you ask me? Mike. Where is the master bedroom? Rea. It's upstairs. Rea points up.Mike.Where is the master bathroom? Rea. It's next to the bedroom. Rea brings her hands together to indicate the relationship between the bedroom and the bathroom.And the house tour continues.

Adults and children spontaneously produce gestures while they speak, and such gestures appear to support and expand on the information communicated by the verbal channel. Little research, however, has been carried out to examine the role played by gesture in the listener's representation of accumulating information. Do listeners attend to the gestures that accompany narrative speech? In what kinds of relationships between gesture and speech do listeners attend to the gestural channel? If listeners do attend to information received in gesture, how is this information represented-is it 'tagged' as originating in the gestural channel? In this article research is described that addresses these questions. Results show that listeners do attend to information conveyed in gesture, when that information supplements or even contradicts the information conveyed by speech. And information received via gesture is available for retelling in speech. These results are taken to demonstrate that gesture is not taken by the listener to be epiphenomenal to the act of speaking, or a simple manual translation of speech. But they also suggest that the information conveyed in a discourse may be represented in a manner that is neither gesture nor language, although accessible to both channels.Pantomime without discourse will leave you nearly tranquil, discourse without gestures will wring tears from you.

In this paper, we argue for embodied corrversational characters as the logical extension of the metaphor of human -computer interaction as a conversation. We argue that the only way to fully model the richness of human I&+ to-face communication is to rely on conversational analysis that describes sets of conversational behaviors as fi~lfilling conversational functions, both interactional and propositional. We demonstrate how to implement this approach in Rea, an embodied conversational agent that is capable of both multimodal input understanding and output generation in a limited application domain. Rea supports both social and task-oriented dialogue. We discuss issues that need to be addressed in creating embodied conversational agents, and describe the architecture of the Rea interface.

No abstract

Fantasy play and storytelling serve an important role in young children's development. While computers are increasingly present in the world of young children, there is a lack of computational tools to support children's voices in everyday storytelling, particularly in the context of fantasy play. We believe that there is a need for computational systems that engage in story-listening rather than storytelling. This paper introduces StoryMat, a system that supports and listens to children's voices in their own storytelling play. StoryMat offers a child-driven, story-listening space by recording and recalling children's narrating voices, and the movements they make with their stuffed animals on a colourful story-evoking quilt. Empirical research with children shows that StoryMat fosters developmentally advanced forms of storytelling of the kind that has been shown to provide a bridge to written literacy, and provides a space where children engage in fantasy storytelling collaboratively with or without a playmate. The paper addresses the importance of supporting young children's fantasy play and suggests a new way for technology to play an integral part in that activity.

Discusses some of the key issues that must be addressed in creating virtual humans, or androids. As a first step, we overview the issues and available tools in three key areas of virtual human research: face-to-face conversation, emotions and personality, and human figure animation. Assembling a virtual human is still a daunting task, but the building blocks are getting bigger and better every day. This material is posted here with permission of the IEEE. Such permission of the IEEE does not in any way imply IEEE endorsement of any of the University of Pennsylvania's products or services. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to pubs-permissions@ieee.org. By choosing to view this document, you agree to all provisions of the copyright laws protecting it. Building a virtual human is a multidisciplinary effort, joining traditional artificial intelligence problems with a range of issues from computer graphics to social science. Virtual humans must act and react in their simulated environment, drawing on the disciplines of automated reasoning and planning. To hold a conversation, they must exploit the full gamut of natural language processing research, from speech recognition and natural language understanding to natural language generation and speech synthesis. Providing human bodies that can be controlled in real time delves into computer graphics and animation. And because an agent looks like a human, people expect it to behave like one as well and will be disturbed by, or misinterpret, discrepancies from human norms. Thus, virtual human research must draw heavily on psychology and communication theory to appropriately convey nonverbal behavior, emotion, and personality. Comments Author(s)Jonathan GratchThis broad range of requirements poses a serious problem. Researchers working on particular aspects of virtual humans cannot explore their component in the context of a complete virtual human unless they can understand results across this array of disciplines and assemble the vast range of software tools (for example, speech recognizers, planners, and animation systems) required to construct one. Moreover, these tools were rarely designed to interoperate and, worse, were often designed with different purposes in mind. For example, most computer graphics research has focused on high fidelity offline image rendering that does not support the fine-grained interactive control that a virtual human must have over its body.In the spring of 2002, about 30 international researchers from across disciplines convened at the University of Southern California to begin to bridge this gap in knowledge and tools (see www.ict.usc.edu/~vhumans). Our ultimate goal is a modular architecture and interface standards that will allow researchers in this area to reuse each other's work. This goal can only be achieve...

We investigate the verbal and nonverbal means for grounding, and propose a design for embodied conversational agents that relies on both kinds of signals to establish common ground in human-computer interaction. We analyzed eye gaze, head nods and attentional focus in the context of a direction-giving task. The distribution of nonverbal behaviors differed depending on the type of dialogue move being grounded, and the overall pattern reflected a monitoring of lack of negative feedback. Based on these results, we present an ECA that uses verbal and nonverbal grounding acts to update dialogue state.