Emotion Expression of Avatar through Eye Behaviors, Lip Synchronization and MPEG4 in Virtual Reality based on Xface Toolkit: Present and Future

Basori, Ahmad Hoirul; Ali, Itimad Raheem

doi:10.1016/j.sbspro.2013.10.290

Cited by 4 publications

(4 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Facial expression is one of the important characteristics of the human-computer interaction [21]. Facial expressions are the patterns involving certain changes in the facial muscles on the forehead, brow, nose, cheek, oral, lips etc.…”

Section: Emotion Space Designmentioning

confidence: 99%

See 1 more Smart Citation

A virtual perception method for urban noise: The calculation of noise annoyance threshold and facial emotion expression in the virtual noise scene

2015

View full text Add to dashboard Cite

Section: Emotion Space Designmentioning

confidence: 99%

“…On the other hand, within application conducted in immersive virtual environments, the facial emotion expression is an important parameter [21,22]. Moreover, the virtual reality technology can be combined with acoustic simulation [23,24].…”

Section: Introductionmentioning

confidence: 99%

A virtual perception method for urban noise: The calculation of noise annoyance threshold and facial emotion expression in the virtual noise scene

2015

View full text Add to dashboard Cite

“…McDonnell et al [22] demonstrated that users can accept motion artifacts in the design of VAs if they are more cartoon-like than realisticlooking (human-like) in appearance. As the lip-sync technique that is commonly used in the research is synthetic to mimic the natural lip shapes (e.g., [23]), we are interested in exploring the relevance of VA realism or naturalism on the user's perception of removing lip-syncing by testing the presence of lip-syncing with both a more realistic-looking VA versus a cartoon-like VA. Hence, in this paper, we investigate the impact of VA's realism and lip-syncing in the context of assisting psychology students to gain knowledge and confidence in their future roles as practicing therapists in the knowledge domain of emotion regulation.…”

Section: Introductionmentioning

confidence: 99%

Do Not Freak Me Out! The Impact of Lip Movement and Appearance on Knowledge Gain and Confidence

Abdulrahman,

Hopman,

Richards

2024

MTI

View full text Add to dashboard Cite

Virtual agents (VAs) have been used effectively for psychoeducation. However, getting the VA’s design right is critical to ensure the user experience does not become a barrier to receiving and responding to the intended message. The study reported in this paper seeks to help first-year psychology students to develop knowledge and confidence to recommend emotion regulation strategies. In previous work, we received negative feedback concerning the VA’s lip-syncing, including creepiness and visual overload, in the case of stroke patients. We seek to test the impact of the removal of lip-syncing on the perception of the VA and its ability to achieve its intended outcomes, also considering the influence of the visual features of the avatar. We conducted a 2 (lip-sync/no lip-sync) × 2 (human-like/cartoon-like) experimental design and measured participants’ perception of the VA in terms of eeriness, user experience, knowledge gain and participants’ confidence to practice their knowledge. While participants showed a tendency to prefer the cartoon look over the human look and the absence of lip-syncing over its presence, all groups reported no significant increase in knowledge but significant increases in confidence in their knowledge and ability to recommend the learnt strategies to others, concluding that realism and lip-syncing did not influence the intended outcomes. Thus, in future designs, we will allow the user to switch off the lip-sync function if they prefer. Further, our findings suggest that lip-syncing should not be a standard animation included with VAs, as is currently the case.

show abstract

“…This renders the dynamics of social perception within SVEs an important research topic in itself (Fabri, Moore, and Hobbs, 2004;Blascovich and Bailenson, 2011;Sun and Won, 2021;Hepperle, Purps, Deuchler, and Wölfel, 2022;Rogers et al, 2022). While various studies have shown that the non-verbal behavior of VCs, including movements, postures, gaze, and facial expressions produce similar effects as real-life or videotaped human behavior Noël, Dumoulin, and Lindgaard, 2009;Basori and Ali, 2013), surprisingly little is known about the specific impact of immersive VR display technology on social perception. Specifically, we do not know whether the subtle emotional nuances in non-verbal communication can be more effectively conveyed in VR if observers and virtual others are co-located in an immersive SVE compared to a desktop VR setting (see .…”

Section: Introductionmentioning

confidence: 99%

Measuring the effects of co-location on emotion perception in shared virtual environments: An ecological perspective

Bente

Schmälzle

Jahn

et al. 2023

Front. Virtual Real.

View full text Add to dashboard Cite

Inferring emotions from others’ non-verbal behavior is a pervasive and fundamental task in social interactions. Typically, real-life encounters imply the co-location of interactants, i.e., their embodiment within a shared spatial-temporal continuum in which the trajectories of the interaction partner’s Expressive Body Movement (EBM) create mutual social affordances. Shared Virtual Environments (SVEs) and Virtual Characters (VCs) are increasingly used to study social perception, allowing to reconcile experimental stimulus control with ecological validity. However, it remains unclear whether display modalities that enable co-presence have an impact on observers responses to VCs’ expressive behaviors. Drawing upon ecological approaches to social perception, we reasoned that sharing the space with a VC should amplify affordances as compared to a screen display, and consequently alter observers’ perceptions of EBM in terms of judgment certainty, hit rates, perceived expressive qualities (arousal and valence), and resulting approach and avoidance tendencies. In a between-subject design, we compared the perception of 54 10-s animations of VCs performing three daily activities (painting, mopping, sanding) in three emotional states (angry, happy, sad)—either displayed in 3D as a co-located VC moving in shared space, or as a 2D replay on a screen that was also placed in the SVEs. Results confirm the effective experimental control of the variable of interest, showing that perceived co-presence was significantly affected by the display modality, while perceived realism and immersion showed no difference. Spatial presence and social presence showed marginal effects. Results suggest that the display modality had a minimal effect on emotion perception. A weak effect was found for the expression “happy,” for which unbiased hit rates were higher in the 3D condition. Importantly, low hit rates were observed for all three emotion categories. However, observers judgments significantly correlated for category assignment and across all rating dimensions, indicating universal decoding principles. While category assignment was erroneous, though, ratings of valence and arousal were consistent with expectations derived from emotion theory. The study demonstrates the value of animated VCs in emotion perception studies and raises new questions regarding the validity of category-based emotion recognition measures.

show abstract

Emotion Expression of Avatar through Eye Behaviors, Lip Synchronization and MPEG4 in Virtual Reality based on Xface Toolkit: Present and Future

Cited by 4 publications

References 17 publications

A virtual perception method for urban noise: The calculation of noise annoyance threshold and facial emotion expression in the virtual noise scene

A virtual perception method for urban noise: The calculation of noise annoyance threshold and facial emotion expression in the virtual noise scene

Do Not Freak Me Out! The Impact of Lip Movement and Appearance on Knowledge Gain and Confidence

Measuring the effects of co-location on emotion perception in shared virtual environments: An ecological perspective

Contact Info

Product

Resources

About