Sound source localization in virtual reality with ambisonics sound reproduction

Abstract

To reproduce realistic audio-visual scenarios in the laboratory, ambisonics is often used to reproduce a sound field over loudspeakers and virtual reality (VR) glasses are used to present visual information. Both technologies have been shown to be suitable for research. However, the combination of both technologies, ambisonics and VR glasses, might affect the spatial cues for auditory localization and thus, the localization percept. Here, we investigated how VR glasses affect the localization of virtual sound sources on the horizontal plane produced using either 1st, 3rd, 5th or 11th order ambisonics with and without visual information. Results showed that with 1st order ambisonics the localization error is larger than with the higher orders, while the differences across the higher orders were small.The physical presence of the VR glasses without visual information increased the perceived lateralization of the auditory stimuli by on average about 2°, especially in the right hemisphere. Presenting visual information about the environment and potential sound sources did reduce this HMD-induced shift, however it could not fully compensate for it. While the localization performance itself was affected by the ambisonics order, there was no interaction between the ambisonics order and the effect of the HMD. Thus, the presence of VR glasses can alter acoustic localization when using ambisonics sound reproduction, but visual information can compensate for most of the effects. As such, most use cases for VR will be unaffected by these shifts in the perceived location of auditory stimuli.

Perceptual evaluation of an ambisonic auralization system of measured 3D acoustics

“…These degradations are manifested through an increase of localization errors in azimuth and elevation as well as an increase of the apparent source widths. These results are comparable to those found in the literature for different ambisonic systems [16,17,40]. Hence, the degradation of azimuthal localization may be attributed to a poor reproduction of binaural cues (ILD and ITD), which are essential for localization in the azimuthal plane.…”

“…On the other hand, wearing a head-mounted device, such as a VR headset, can have an impact on the HRTF of the listener wearing it [39]. Recently, Huisman et al investigated the localization performance with a VR headset for 2D ambisonic sound reproduction at different orders, with and without visual feedback [40]. In their study, the localization task was performed in an open-loop fashion (i.e.…”

“…As elevation perception relies on monaural frequency cues within the 4À16 kHz frequency range, the spatial aliasing at high frequencies, imposed by the characteristics of the microphone array (above 5 kHz for the em32) can explain the difficulty in locating the sources along the vertical plane. Moreover, while wearing a VR headset seems to have a limited impact on azimuthal localization performance during 2D ambisonic rendering [40], it may have a more significant effect on the perception of source elevation.…”

Spatial Resolution of Late Reverberation in Virtual Acoustic Environments

“…While artificial reverberation has historically been used as a tool for artistic expression (Välimäki et al, 2012), room acoustics simulations are also utilized for planning of spaces (e.g., concert halls and classrooms; Rindel, 2001). Recently, room acoustics simulations and virtual acoustic environments (VAEs) have gained interest as tools for psychoacoustic research and hearing aid development (Ahrens et al, 2019;Grimm et al, 2019;Huisman et al, 2019;Pausch & Fels, 2020;Pausch & Fels, 2020;Rindel, 2001;Schutte et al, 2019;Seeber et al, 2010), particularly in the context of ecological validity, where the results of speech-in-noise and hearing test with simplified "synthetic" stimuli have been questioned with regard to real-world outcomes (Cord et al, 2007;Jerger, 2009;Keidser et al, 2020;Miles et al, 2020).…”

Ambisonics Sound Source Localization With Varying Amount of Visual Information in Virtual Reality

“…The model of the room and loudspeaker array used in the experiment was created by Kasper Duemose Lund. This work was shared on a preprint server (Huisman et al, 2021).…”