From Greta's mind to her face: modelling the dynamics of affective states in a conversational embodied agent

Rosis, Fiorella de; Pélachaud, Catherine; Poggi, Isabella; Carofiglio, Valeria; Carolis, Berardina De

doi:10.1016/s1071-5819(03)00020-x

Cited by 209 publications

(117 citation statements)

References 50 publications

Supporting

Mentioning

106

Contrasting

Unclassified

Order By: Relevance

“…Her background in epidemiology allowed her to import many insights from the medical perspective to user modeling, and her work on public health informatics was well ahead of its time: hers were some of the early works on producing personalised patient information (Rosis and Grasso 1995). She was a leader in the field of intelligent interfaces, and her work on modelling affect and emotion in embodied agents was internationally recognised (de Rosis 2003). She died of cancer in July 2008.…”

Section: In Memoriammentioning

confidence: 99%

Preface to the special issue on personalization for e-health

Grasso

Paris

2011

User Model User-Adap Inter

View full text Add to dashboard Cite

Section: In Memoriammentioning

confidence: 99%

Preface to the special issue on personalization for e-health

Grasso

Paris

2011

User Model User-Adap Inter

View full text Add to dashboard Cite

“…The virtual agent systems, Improv (Perlin and Goldberg, 1996), AESOPWORLD (Okada et al, 1999), Behaviour Generation System (Breitfuss et al, 2007), Max (Kopp et al, 2008) and Greta (Pelachaud, 2005 andDe Rosis et al, 2003) investigate in great detail, how to model an agent's mind with sophisticated reasoning about emotional states, personality, social and conversational roles. Furthermore, these systems connect those mental processes to facial, gestural and vocal expressions of 3D models achieving human-like conversational behaviour.…”

Section: Relation To Other Workmentioning

confidence: 99%

SceneMaker: Intelligent Multimodal Visualisation of Natural Language Scripts

Hanser

Kevitt

Lunney

et al. 2010

Artificial Intelligence and Cognitive Science

View full text Add to dashboard Cite

Performing plays or creating films and animations is a complex creative, and thus expensive, process involving various professionals and media. This research project proposes to augment this process by automatically interpreting film and play scripts and automatically generating animated scenes from them. For this purpose a web based software prototype, SceneMaker, will be developed. During the generation of the story content, special attention will be given to emotional aspects and their reflection in the execution of all types of modalities: fluency and manner of actions and behaviour, body language, facial expressions, speech, voice pitch, scene composition, timing, lighting, music and camera. The main objective of this work is to demonstrate how a scene and actor behaviour changes, when emotional states are taken into account, e.g. walking down a street in a happy versus a sad state. Consequently, more realistic and believable story visualisations are an expected outcome. Literature on related research areas is reviewed involving natural language and text, screenplay or play script, layout processing with regard to personality and emotion detection, modelling affective behaviour of embodied agents, visualisation of 3D scenes with digital cinematography and genre-based presentation, intelligent multimedia selection and mobile 3D technology.SceneMaker's architecture with a language and text analysis module, a reasoning and decision making module based on cognitive, emotional information and a multimedia module for multimodal visualisation are presented along with a development plan and test scenarios. Technologies and software relevant for the development of SceneMaker are analysed and prospective tools for text and language processing, visualisation, media allocation and mobile user interfaces are suggested. Accuracy of content animation, effectiveness of expression and usability of the interface will be evaluated in empirical tests. In relation to other work, this project will present a genre-specific text-to-animation methodology which combines all relevant expressive modalities. Emotional expressivity inspired by the OCC (Ortony, Clore and Collins) emotion model will influence all modalities to enhance believability of virtual actors as well as scene presentation. Compared to other scene production systems, SceneMaker will infer emotions from the story context, rather than relying on explicit emotion keywords, SceneMaker will automatically detect genre from the script and apply appropriate cinematic direction. Also SceneMaker will enable 3D animation editing via web-based and mobile platforms.In conclusion, SceneMaker will reduce production time, save costs and enhance communication of ideas through the development of an intelligent, multimodal animation generation system providing quick previsualisations of scenes from script text input.

show abstract

“…However, relevant research is also being carried out in a range of individual disciplines, including believable facial, gesture and body animation of virtual characters [4], modeling personality and emotion [5], expressive speech synthesis [6] and control mechanisms for story and interaction [7].…”

Section: Motivationmentioning

confidence: 99%

Enhancing Animated Agents in an Instrumented Poker Game

Schröder

Gebhard

Charfuelàn

et al. 2008

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. In this paper we present an interactive poker game in which one human user plays against two animated agents using RFID-tagged poker cards. The game is used as a showcase to illustrate how current AI technologies can be used for providing new features to computer games. A powerful and easy-to-use multimodal dialog authoring tool is used for modeling game content and interaction. The poker characters rely on a sophisticated model of affect and a state-of-the art speech synthesizer. Through the combination of these methods, the characters show a consistent expressive behavior that enhances the naturalness of interaction in the game. MotivationAn application area with growing relevance for AI technologies are computer games. Specifically, driving the behavior of non-player characters raises a whole range of challenges, such as interaction design, emotion modeling, figure animation, and speech synthesis [1].Some research projects address these problems in combination, such as the interactive drama game Façade [2] or the Mission Rehearsal Exercise Project [3]. However, relevant research is also being carried out in a range of individual disciplines, including believable facial, gesture and body animation of virtual characters [4], modeling personality and emotion [5], expressive speech synthesis [6] and control mechanisms for story and interaction [7].In the project IDEAS4Games, we investigate how modern AI technologies can help to improve the process of creating computer games with interactive expressive virtual characters. Based on the experience of a computer game company (RadonLabs, Berlin), we identified four main challenges in creating computer games: (i) fast creation of game demonstrators; (ii) localization of game content; (iii) easy interaction; and (iv) consistent quality. We address these challenges by combining, in an integrated system, flexible multimodal dialog and story modeling, sophisticated simulation of affect in real time, expressive synthesis, and rendering of 3D virtual characters.We explain the concept of the integrated game demonstrator before providing some details about the individual component technologies.

show abstract

From Greta's mind to her face: modelling the dynamics of affective states in a conversational embodied agent

Cited by 209 publications

References 50 publications

Preface to the special issue on personalization for e-health

Preface to the special issue on personalization for e-health

SceneMaker: Intelligent Multimodal Visualisation of Natural Language Scripts

Enhancing Animated Agents in an Instrumented Poker Game

Contact Info

Product

Resources

About