The effects of spatial auditory and visual cues on mixed reality remote collaboration

Yang, Jing; Sasikumar, Prasanth; Bai, Huidong; Barde, Amit; Sörös, Gábor; Billinghurst, Mark

doi:10.1007/s12193-020-00331-1

Cited by 45 publications

(40 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Auditory notifications are also effective in communicating urgency in driving contexts, which demands high levels of attention from drivers [30]. These types of notifications have also been successfully used to communicate the presence of objects in space when in VR [52].…”

Section: Context-aware Notifications In Mixed Realitymentioning

confidence: 99%

“…In addition to the visual cues, audio cues have been proposed by previous work as a way to get the users' attention in tasks that are mainly visual [40,52]. We were interested in evaluating the users' perception of continuous vs discrete notifications, and if spatial sound was accurate enough to allow users to locate nonimmersed people.…”

Section: Audio Cues: Continuous Vs Discretementioning

confidence: 99%

See 1 more Smart Citation

Promoting Reality Awareness in Virtual Reality through Proxemics

Medeiros

Anjos

Pantidi

et al. 2021

2021 IEEE Virtual Reality and 3D User Interfaces (VR)

View full text Add to dashboard Cite

Head-Mounted Virtual reality (VR) systems provide full-immersive experiences to users and completely isolate them from the outside world, placing them in unsafe situations. Existing research proposed different alert-based solutions to address this. Our work builds on these studies on notification systems for VR environments from a different perspective. We focus on: (i) exploring alert systems to notify VR users about non-immersed bystanders' in socially related, non-critical interaction contexts; (ii) understanding how best to provide awareness of non-immersed bystanders while maintaining presence and immersion within the Virtual Environment(VE). To this end, we developed single and combined alert cues -leveraging proxemics, perception channels, and push/pull approaches and evaluated those via two user studies. Our findings indicate a strong preference towards maintaining immersion and combining audio and visual cues, push and pull notification techniques that evolve dynamically based on proximity.

show abstract

Section: Context-aware Notifications In Mixed Realitymentioning

confidence: 99%

Section: Audio Cues: Continuous Vs Discretementioning

confidence: 99%

Promoting Reality Awareness in Virtual Reality through Proxemics

Medeiros

Anjos

Pantidi

et al. 2021

2021 IEEE Virtual Reality and 3D User Interfaces (VR)

View full text Add to dashboard Cite

show abstract

“…Most MR training systems have explored the use of visual cues in the forms of graphic objects or text to deliver instructions. For example, Yang et al (2020) compared the impact of spatial audio in remote collaboration among various visual cues like hand gestures, annotation and eye gaze. There has also been some research using point-clouds to visualize instructional cues from trainers, but it is not well studied (Schwald and de Laval (2003); Ke et al (2005); Olwal et al (2008); Alem and Li (2011)).…”

Section: Introductionmentioning

confidence: 99%

Spatial Perception Enhancement in Assembly Training Using Augmented Volumetric Playback

Sasikumar

Chittajallu

Raj

et al. 2021

Front. Virtual Real.

Self Cite

View full text Add to dashboard Cite

Conventional training and remote collaboration systems allow users to see each other’s faces, heightening the sense of presence while sharing content like videos or slideshows. However, these methods lack depth information and a free 3D perspective of the training content. This paper investigates the impact of volumetric playback in a Mixed Reality (MR) spatial training system. We describe the MR system in a mechanical assembly scenario that incorporates various instruction delivery cues. Building upon previous research, four spatial instruction cues were explored; “Annotation”, “Hand gestures”, “Avatar”, and “Volumetric playback”. Through two user studies that simulated a real-world mechanical assembly task, we found that the volumetric visual cue enhanced spatial perception in the tested MR training tasks, exhibiting increased co-presence and system usability while reducing mental workload and frustration. We also found that the given tasks required less effort and mental load when eye gaze was incorporated. Eye gaze on its own was not perceived to be very useful, but it helped to compliment the hand gesture cues. Finally, we discuss limitations, future work and potential applications of our system.

show abstract

“…Recent advances in technology have resulted in VR and augmented reality (AR) establishing a foothold in the consumer market. Research in the HCI domain has demonstrated the usefulness of these technologies to mediate remote collaboration [16][17][18][19]. Collaboration in virtual environments (VE) has been shown to be nearly as effective as that in face-to-face situations in the real world [16,18,19].…”

Section: Introductionmentioning

confidence: 99%

“…Research in the HCI domain has demonstrated the usefulness of these technologies to mediate remote collaboration [16][17][18][19]. Collaboration in virtual environments (VE) has been shown to be nearly as effective as that in face-to-face situations in the real world [16,18,19]. The advantage of VEs is that interaction techniques in such environments can be tailored in order to implement optimal interactions between the collaborators and the environment [20].…”

Section: Introductionmentioning

confidence: 99%

A Review of Hyperscanning and Its Use in Virtual Environments

et al. 2020

Self Cite

View full text Add to dashboard Cite

Hyperscanning is a technique which simultaneously records the neural activity of two or more people. This is done using one of several neuroimaging methods, such as electroencephalography (EEG), functional magnetic resonance imaging (fMRI), and functional near-infrared spectroscopy (fNIRS). The use of hyperscanning has seen a dramatic rise in recent years to monitor social interactions between two or more people. Similarly, there has been an increase in the use of virtual reality (VR) for collaboration, and an increase in the frequency of social interactions being carried out in virtual environments (VE). In light of this, it is important to understand how interactions function within VEs, and how they can be enhanced to improve their quality in a VE. In this paper, we present some of the work that has been undertaken in the field of social neuroscience, with a special emphasis on hyperscanning. We also cover the literature detailing the work that has been carried out in the human–computer interaction domain that addresses remote collaboration. Finally, we present a way forward where these two research domains can be combined to explore how monitoring the neural activity of a group of participants in VE could enhance collaboration among them.

show abstract

The effects of spatial auditory and visual cues on mixed reality remote collaboration

Cited by 45 publications

References 40 publications

Promoting Reality Awareness in Virtual Reality through Proxemics

Promoting Reality Awareness in Virtual Reality through Proxemics

Spatial Perception Enhancement in Assembly Training Using Augmented Volumetric Playback

A Review of Hyperscanning and Its Use in Virtual Environments

Contact Info

Product

Resources

About