Statistical Learning of Facial Expressions Improves Realism of Animated Avatar Faces

Abstract

A high realism of avatars is beneficial for virtual reality experiences such as avatar-mediated communication and embodiment. Previous work, however, suggested that the usage of realistic virtual faces can lead to unexpected and undesired effects, including phenomena like the uncanny valley. This work investigates the role of photographic and behavioral realism of avatars with animated facial expressions on perceived realism and congruence ratings. More specifically, we examine ratings of photographic and behavioral realism and their mismatch in differently created avatar faces. Furthermore, we utilize these avatars to investigate the effect of behavioral realism on perceived congruence between video-recorded physical person’s expressions and their imitations by the avatar. We compared two types of avatars, both with four identities that were created from the same facial photographs. The first type of avatars contains expressions that were designed by an artistic expert. The second type contains expressions that were statistically learned from a 3D facial expression database. Our results show that the avatars containing learned facial expressions were rated more photographically and behaviorally realistic and possessed a lower mismatch between the two dimensions. They were also perceived as more congruent to the video-recorded physical person’s expressions. We discuss our findings and the potential benefit of avatars with learned facial expressions for experiences in virtual reality and future research on enfacement.

Avatars with faces of real people: A construction method for scientific experiments in virtual reality

“…The faces of our avatars are also static and thus cannot currently convey expressions or articulate speech. Facial rigging is challenging (Grewe et al, 2021;Lewis et al, 2014) and, if poorly implemented, can reduce the perception of behavioural realism (Grewe et al, 2021). Advances in technology and expertise will close this gap, to create avatars that not only visually resemble their real-life counterparts, but which incorporate their speech and motion patterns as well.…”

Development of self-regulated lifelong learning (SR3Ls) model in the era of IR4.0 for post-pandemic economy

“…Though much work has been done to improve photographic and behavioural realism of the animated avatar faces, the realistic facial expressions are still superficial and less likely to establish humanlike dynamic engagement. The characters of avatars available are also limited to few identities (Grewe et al ., 2021).…”

Expressiveness of real-time motion captured avatars influences perceived animation realism and perceived quality of social interaction in virtual reality

“…Avatars are the visual representation of social actors within the virtual environment (Kyrlitsias and Michael-GrigoriouLugrin et al, 2015;Seymour et al, 2018;. Two key qualities of avatars include their visual depiction and their movement capabilities (Grewe et al, 2021;Higgins et al, 2022;Latoschik et al, 2017;Stuart et al, 2022;Wu et al, 2021). While we recognize the importance of both these qualities, in our research we are focused on the visual expressiveness of the avatar.…”

“…Motion capture technologies can affect the experience of social VR because motion captured avatars are able to convey a wider range of social cues during an interaction (Kokkinara and McDonnell, 2015;Lugrin et al, 2015;Latoschik et al, 2017;Herrera et al, 2020;Kyrlitsias and Michael-Grigoriou, 2022;Rogers et al, 2022). For instance, avatars that express interest with direct eye gaze to our presence and behaviour in VR are often rated as having a higher level of behavioural realism (Aburumman et al, 2022;Grewe et al, 2021;Wu et al, 2021).…”