Comparing technologies for conveying emotions through realistic avatars in virtual reality‐based metaverse experiences

Abstract

With the development of metaverse(s), industry and academia are searching for the best ways to represent users' avatars in shared virtual environments (VEs), where real‐time communication between users is required. The expressiveness of avatars is crucial for transmitting emotions that are key for social presence and user experience, and are conveyed via verbal and non‐verbal facial and body signals. In this paper, two real‐time modalities for conveying expressions in virtual reality (VR) via realistic, full‐body avatars are compared by means of a user study. The first modality uses dedicated hardware (i.e., eye and facial trackers) to allow a mapping between the user's facial expressions/eye movements and the avatar model. The second modality relies on an algorithm that, starting from an audio clip, approximates the facial motion by generating plausible lip and eye movements. The participants were requested to observe, for both the modalities, the avatar of an actor performing six scenes involving as many basic emotions. The evaluation considered mainly social presence and emotion conveyance. Results showed a clear superiority of facial tracking when compared to lip sync in conveying sadness and disgust. The same was less evident for happiness and fear. No differences were observed for anger and surprise.

Embedding Emotions in the Metaverse: The Emotive Keywords for Augmented Reality Mobile Library Application

“…It emerges from a combination of numerous technologies that blend together with the ability to expand the physical world through augmented reality (AR) and virtual reality (VR) [20] offering a fully dynamic and immersive virtual environment [21]. The application of the metaverse extends across diverse sectors, including education for innovative libraries [22], [23], enhanced campus and classroom experiences [24]- [26], retail to elevate shopping experiences [27], [28], games for revolutionary entertainment [29], [30], and the medical field to enhance healthcare [31], [32]. It is expected to be applied in an even broader range of sectors in the future.…”

Feasibility study of virtual reality in audiovisual environment: Assessment of university cafeteria acoustic environment

“…5 It is worth noting that in the process of visual and auditory interaction, the satisfaction of visual experience, haptic sensation, and auditory experience are crucial factors in determining the level of immersion and realism in the virtual environment. 6,7 Ferzana Chaze successfully explored the potential of elderly individuals' physical, emotional, cognitive, and social well-being using highly realistic VR scenarios. 8 However, visual elements often dominate the scene, while sound sources serve as auxiliary tools during the interaction.…”

“…Today, virtual reality technology is supported by various game engines and renderers, 4 such as Unreal Engine, Unity3D, Vray, Enscape, and others, which provide highly immersive experiences for users 5 . It is worth noting that in the process of visual and auditory interaction, the satisfaction of visual experience, haptic sensation, and auditory experience are crucial factors in determining the level of immersion and realism in the virtual environment 6,7 . Ferzana Chaze successfully explored the potential of elderly individuals' physical, emotional, cognitive, and social well‐being using highly realistic VR scenarios 8 .…”