An Eye Gaze Model for Controlling the Display of Social Status in Believable Virtual Humans

Abstract

Designing highly believable characters remains a major concern within digital games. Matching a chosen personality and other dramatic qualities to displayed behavior is an important part of improving overall believability. Gaze is a critical component of social exchanges and serves to make characters engaging or aloof, as well as to establish character's role in a conversation.In this paper, we investigate the communication of status related social signals by means of a virtual human's eye gaze. We constructed a cross-domain verbal-conceptual computational model of gaze for virtual humans to facilitate the display of social status. We describe the validation of the model's parameters, including the length of eye contact and gazes, movement velocity, equilibrium response, and head and body posture. In a first set of studies, conducted on Amazon Mechanical Turk using prerecorded video clips of animated characters, we found statistically significant differences in how the characters' status was rated based on the variation in social status.In a second step based on these empirical findings, we designed an interactive system that incorporates dynamic eye tracking and spoken dialog, along with real-time control of a virtual character. We evaluated the model using a presential, interactive scenario of a simulated hiring interview. Corroborating our previous finding, the interactive study yielded significant differences in perception of status were found (p = .046). Thus, we believe status is an important aspect of dramatic believability, and accordingly, this paper presents our social eye gaze model for realistic procedurally animated characters and shows its efficacy.

The Rise of Interactional Multimodality in Human-Computer Interaction

“…Beyond the more modest strategies of implementing at least some listening behavior, research has increasingly shifted focus to implementations of higher-order interactional features, such as social status-specific multimodal conduct [Nixon, DiPaola, Bernardet, 2018] and emotion monitoring, classifying and mirroring [Yalçın, 2020]. The general trend can be represented by a hierarchy (see fig.…”

Modeling Empathy in Embodied Conversational Agents

“…The behavior generation component allows for synchronization of reflective behavior such as shifting body posture, breathing gaze or self-touching behavior as responses to the events happening in the environment and idle movement patterns during the lack of external input increases the quality of the interaction (Bernardet, Kang, Feng, DiPaola, & Shapiro, 2017;Nixon, DiPaola, & Bernardet, 2018). The consistency and coordination of these movements allow generating a sense of personality in affective agents (Bernardet, Saberi, & DiPaola, 2016).…”

“…The posture of the users, head gestures and hand gestures would provide valuable information about the personality of the user as well as the context and emotions of the user (DeVault et al, 2014). The personal space the user is keeping with the agent (Saberi, Bernardet, & DiPaola, 2015 as well as the gaze behavior (Nixon et al, 2018) can indicate to the social context and personality. Moreover, additional biosensing capabilities such as heart-beat, skin conductance, breathing patterns as well as EEG signals can open up new possibilities of how an interaction between an empathic agent could look like, by giving additional information on the user.…”