47Published by the IEEE Computer Society I n t e r a c t i v e E n t e r t a i n m e n t
Presenting in Virtual Worlds:
An Architecture for a 3D Anthropomorphic PresenterHerwin van Welbergen, Anton Nijholt, Dennis Reidsma, and Job Zwiers, University of Twente M eeting and lecture room technology is a burgeoning field. Such technology can provide real-time support for physically present participants, for online remote participation, or for offline access to meetings or lectures. Capturing relevant information from meetings or lectures is necessary to provide this kind of support.Multimedia presentation of this captured information requires a lot of attention.Our previous research has looked at including in these multimedia presentations a regeneration of meeting events and interactions in virtual reality. We developed technology that translates captured meeting activities into a virtual-reality version that lets us add and manipulate information. 1 In that research, our starting point was the human presenter or meeting participant. Here, it's a semiautonomous virtual presenter that performs in a virtual-reality environment (see figure 1). The presenter's audience might consist of humans, humans represented by embodied virtual agents, and autonomous agents that are visiting the virtual lecture room or have roles in it.In this article, we focus on models and associated algorithms that steer the virtual presenter's presentation animations. In our approach, we generate the presentations from a script describing the synchronization of speech, gestures, and movements. The script has also a channel devoted to presentation sheets (slides) and sheet changes, which we assume are an essential part of the presentation. This channel can also present material other than sheets, such as annotated paintings or movies.
The virtual presenter's architectureBuilding a virtual presenter involves many different techniques, including facial and body animation and speech, emotion, and presentation style generation. The main challenge is to integrate those elements in a single virtual human.
Integration concernsSuch integration raises two major concerns. 2 The first is consistency. When an agent's internal state (for example, goals, plans, and emotions) as well as the various channels of outward behavior (such as speech, body movement, and facial expressions) are in conflict, inconsistency arises. The agent might then look clumsy or awkward, or, even worse, appear confused, conflicted, emotionally detached, repetitious, or simply fake. Because our virtual presenter currently derives its behavior from the annotated script of a real presentation, consistency conflicts arise mostly between the implemented and unimplemented channels. For example, one of our 3D models can't move its mouth. When it speaks, this looks awkward. When we extend the presenter to dynamically generate its behavior, consistency will become even more important.The second concern, timing, is currently more crucial. The agent's different output channels should be proper...