Multimodal Cues of the Sense of Presence and Co-presence in Human-Virtual Agent Interaction

Abstract

A key challenge when studying human-agent interaction is the evaluation of user's experience. In virtual reality, this question is addressed by studying the sense of "presence" and"co-presence", generally assessed thanks to well-grounded subjective post-experience questionnaires. In this article, we aim at exploring behavioral measures of presence and co-presence by analyzing multimodal cues produced during an interaction both by the user and the virtual agent. In our study, we started from a corpus of human-agent interaction collected in a task-oriented context: a virtual environment aiming at training doctors to break bad news to a patient (played by a virtual agent). Based on this corpus, we have used machine learning algorithms to explore the possibility of predicting user's sense of presence and co-presence. In particular, we have applied and compared two techniques, Random forest and SVM, both showing very good results in predicting the level of presence and co-presence.

Interpretable prediction of brain activity during conversations from multimodal behavioral signals

“…Lexical richness measure [42] Based on time-aligned transcript: (number of adjectives + number of adverbs) / (total number of tokens).…”

Ubiq-Genie: Leveraging External Frameworks for Enhanced Social VR Experiences

“…head rotation velocity and words spoken [17]), communication style (e.g., recognition of F-formations [22]), emotional state (e.g., sentiment analysis using natural-language processing [7]), and user experience (e.g. the objective representations of presence and co-presence [18]). This could be particularly useful in observational, pilot, and remote studies [12,23].…”

Gamifying the Learning Experience Using Extended Reality Technologies for Teaching English as a Foreign Language

“…Immersion is meant to create the illusion that the user has left the actual world and is "present" in the virtual environment. Although immersion is an objective "technology-related" characteristic of nonphysical environments, presence is a psychological, perceptual, and cognitive result of immersion (Ochs et al, 2018). To put it another way, presence might be defined as psychological immersion in a virtual environment.…”

“…In SVREs, users experience a superior sense of self since the individual can control his or her personified agent or avatar . When meeting with other avatars or peers, there is a powerful sense of co-presence (Ochs et al, 2018). In a virtual environment, identification with one's avatar can have a significant psychological impact on behavior and learning .…”