Creating interactive virtual humans: some assembly required

Abstract

Discusses some of the key issues that must be addressed in creating virtual humans, or androids. As a first step, we overview the issues and available tools in three key areas of virtual human research: face-to-face conversation, emotions and personality, and human figure animation. Assembling a virtual human is still a daunting task, but the building blocks are getting bigger and better every day. This material is posted here with permission of the IEEE. Such permission of the IEEE does not in any way imply IEEE endorsement of any of the University of Pennsylvania's products or services. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to pubs-permissions@ieee.org. By choosing to view this document, you agree to all provisions of the copyright laws protecting it. Building a virtual human is a multidisciplinary effort, joining traditional artificial intelligence problems with a range of issues from computer graphics to social science. Virtual humans must act and react in their simulated environment, drawing on the disciplines of automated reasoning and planning. To hold a conversation, they must exploit the full gamut of natural language processing research, from speech recognition and natural language understanding to natural language generation and speech synthesis. Providing human bodies that can be controlled in real time delves into computer graphics and animation. And because an agent looks like a human, people expect it to behave like one as well and will be disturbed by, or misinterpret, discrepancies from human norms. Thus, virtual human research must draw heavily on psychology and communication theory to appropriately convey nonverbal behavior, emotion, and personality. Comments Author(s)Jonathan GratchThis broad range of requirements poses a serious problem. Researchers working on particular aspects of virtual humans cannot explore their component in the context of a complete virtual human unless they can understand results across this array of disciplines and assemble the vast range of software tools (for example, speech recognizers, planners, and animation systems) required to construct one. Moreover, these tools were rarely designed to interoperate and, worse, were often designed with different purposes in mind. For example, most computer graphics research has focused on high fidelity offline image rendering that does not support the fine-grained interactive control that a virtual human must have over its body.In the spring of 2002, about 30 international researchers from across disciplines convened at the University of Southern California to begin to bridge this gap in knowledge and tools (see www.ict.usc.edu/~vhumans). Our ultimate goal is a modular architecture and interface standards that will allow researchers in this area to reuse each other's work. This goal can only be achieve...

Enhancing Animated Agents in an Instrumented Poker Game

“…Specifically, driving the behavior of non-player characters raises a whole range of challenges, such as interaction design, emotion modeling, figure animation, and speech synthesis [1].…”

Models of Culture for Virtual Human Conversation

“…Virtual Humans [1,2] are artificial agents that include both a visual body with a humanlike appearance and range of observable behaviors, and a cognitive component that can make decisions and control the behaviors to engage in human-like activities. Virtual humans often engage in conversation (embodied conversational agents [3]) and can play a number of roles, including providing a human-like interface to information services, acting as a roleplayer in a training system, acting as a tutor, or a "non-player character" in a game.…”

IDEAS4Games: Building Expressive Virtual Characters for Computer Games

“…The creation of interactive expressive characters with a consistent behavior comes with a whole range of challenges, such as interaction design, emotion modeling, figure animation, and speech synthesis [2]. The interactive drama game Façade [3] or the Mission Rehearsal Exercise Project [4] explicitly address these problems in combination, in an integrated application.…”

“…In general, virtual characters have the purpose to enrich the game play experience by showing an engaging and consistent interactive behavior. Because this issue influences acceptance in general, computer games, and virtual characters in particular, have to be designed carefully according to Loyall's suspension of disbelief principle [1]:... a character is considered to be believable if it allows the audience to suspend their disbelief ...The creation of interactive expressive characters with a consistent behavior comes with a whole range of challenges, such as interaction design, emotion modeling, figure animation, and speech synthesis [2]. The interactive drama game Façade [3] or the Mission Rehearsal Exercise Project [4] explicitly address these problems in combination, in an integrated application.…”