Copresence With Virtual Humans in Mixed Reality: The Impact of Contextual Responsiveness on Social Perceptions

Abstract

Virtual humans (VHs)—automated, three-dimensional agents—can serve as realistic embodiments for social interactions with human users. Extant literature suggests that a user’s cognitive and affective responses toward a VH depend on the extent to which the interaction elicits a sense of copresence, or the subjective “sense of being together.” Furthermore, prior research has linked copresence to important social outcomes (e.g., likeability and trust), emphasizing the need to understand which factors contribute to this psychological state. Although there is some understanding of the determinants of copresence in virtual reality (VR) (cf. Oh et al., 2018), it is less known what determines copresence in mixed reality (MR), a modality wherein VHs have unique access to social cues in a “real-world” setting. In the current study, we examined the extent to which a VH’s responsiveness to events occurring in the user’s physical environment increased a sense of copresence and heightened affective connections to the VH. Participants (N = 65) engaged in two collaborative tasks with a (nonspeaking) VH using an MR headset. In the first task, no event in the participant’s physical environment would occur, which served as the control condition. In the second task, an event in the participants’ physical environment occurred, to which the VH either responded or ignored depending on the experimental condition. Copresence and interpersonal evaluations of the VHs were measured after each collaborative task via self-reported measures. Results show that when the VH responded to the physical event, participants experienced a significant stronger sense of copresence than when the VH did not respond. However, responsiveness did not elicit more positive evaluations toward the VH (likeability and emotional connectedness). This study is an integral first step in establishing how and when affective and cognitive components of evaluations during social interactions diverge. Importantly, the findings suggest that feeling copresence with VH in MR is partially determined by the VHs’ response to events in the actual physical environment shared by both interactants.

The impact of virtual agents’ multimodal communication on brain activity and cognitive load in Virtual Reality

“…We then helped the participants put on the VR headset and start the main task. At the end of each session, participants were asked to fill out the Copresence questionnaire (Pimentel and Vinkers, 2021), the Subjective Mental Effort Questionnaire (SMEQ) (Zijlstra and Van Doorn, 1985;Sauro and Dumas, 2009) and the NASA Task Load Index (NASA-TLX) questionnaire (Hart and Staveland, 1988). We also interviewed the participants with open-ended questions to understand their experiences and feelings on the IVA.…”

“…We used the task score logged from each session to measure the task performance. We also collected data using the Copresence questionnaire (Pimentel and Vinkers, 2021), SMEQ (Sauro and Dumas, 2009) and NASA-TLX questionnaire (Hart and Staveland, 1988) to investigate subjective user experience on the feelings of being together with virtual humans and the workload of the collaborative task for each condition. Since the SMEQ has only one question on the overall feeling of cognitive load while the NASA-TLX has six items focusing on both mental and physical efforts to finish the task, we used both to obtain comprehensive understanding on the cognitive load of our task.…”

Effects of Walking Together in a Co-Located Virtual Reality Game

“…When the opponent was able to move the physical token, participants reported higher co-presence and found the opponent more physical. Moreover, Pimentel and Vinkers [17] found that co-presence directed at a virtual human may be higher when the virtual human responds non-verbally to physical events occurring outside the VE. Krum, Kang, and Phan [7] examined how proxemic behavior toward a virtual human is affected by locomotion technique (joystick or physical walking) and haptic priming (in the beginning of the experience, a confederate touched the user when the virtual human did).…”

Application of Graphical Representations From Video Games to Amplify Co-Presence in Group Visits to Virtual Houses

“…Podkosova and Kaufmann (2018) concluded that users try not to collide in the virtual world even when aware that they were safe from real collisions. Pimentel and Vinkers (2021) refer the possibility of co-presence in mixed reality. Kim et al (2018) showed the existence of co-presence between a virtual interlocutor and a human being.…”