On the Relative Importance of Visual and Spatial Audio Rendering on VR Immersion

Abstract

A study was performed using a virtual environment to investigate the relative importance of spatial audio fidelity and video resolution on perceived audio-visual quality and immersion. Subjects wore a head-mounted display and headphones and were presented with a virtual environment featuring music and speech stimuli using three levels each of spatial audio quality and video resolution. Spatial audio was rendered monaurally, binaurally with head-tracking, and binaurally with head-tracking and room acoustic rendering. Video was rendered at resolutions of 0.5 megapixels per eye, 1.5 megapixels per eye, and 2.5 megapixels per eye. Results showed that both video resolution and spatial audio rendering had a statistically significant effect on both immersion and audio-visual quality. Most strikingly, the results showed that under the conditions that were tested in the experiment, the addition of room acoustic rendering to head-tracked binaural audio had the same improvement on immersion as increasing the video resolution five-fold, from 0.5 megapixels per eye to 2.5 megapixels per eye.

The Effect of Training on Localizing HoloLens-Generated 3D Sound Sources

“…There are other measures such as the Mahalanobis distance [34] and KL divergence [35] available for use to define the distance between two statistical distributions instead of d as defined in Equation (2). Note that d is similar to the t-value used for the paired t-test except that µ 1 − µ 2 in d is a signed number to represent the increase/decrease in performance, while no sample size is incorporated into σ 1 and σ 2 .…”

“…This suggests that the current spatial audio system of the HoloLens 2 may not provide sufficient cues for accurate elevation perception and need improvement in its design. (2) We recognized the need to train HoloLens 2 users somehow each time when they started to use the device, so that they could adapt to sound localization with sufficient accuracy. This is based on our finding that the user performance in sound localization was not satisfactory when they started to use the device during the two weeks of training sessions designed for observing the temporal retention of training effect.…”

“…In a VR environment, it is crucial to synchronize the 3D sound localization with the user’s visual experience. When the auditory cues align seamlessly with the visual cues, listeners not only feel more immersed, but also become more attuned to the VR encounter [ 2 ].…”

Digitization, Preservation, and Dance Narrative Exploration of Red Music Cultural Heritage in Hebei Province: A Multidisciplinary Approach

“…Through VR headsets, individuals can virtually attend concerts, operas, and festivals, thereby gaining deeper insights into the contextual and atmospheric nuances associated with these events. The interactive nature of VR further allows learners to engage with meticulously crafted digital replicas of traditional instruments, facilitating hands-on exploration of their design, functionality, and auditory qualities, thereby fostering a more profound understanding of the music's material culture [8].…”

User Expectation of Room Acoustic Parameters in Virtual Reality Environments

“…This allowed for real-time rendering of audio within an accurate room model, as is required in a six-degrees-of-freedom (6DOF) VR environment. SDN has been shown to result in (a) a higher "naturalness" than feedback delay networks (FDNs) [11], convolution methods and ray-tracing [12], (b) a greater sense of externalisation than higher-order ambisonics measurements [13], and (c) a greater sense of immersion in VR environments [14]. Furthermore, it was successfully incorporated in various realtime binaural rendering applications [13], [15], [16], and it has since been extended to (a) model outdoor scenes [17], (b) coupled enclosures [18], and (c) exact modelling of higherorder reflections [19].…”