Short-term effects of sound localization training in virtual reality

Abstract

Head-related transfer functions (HRTFs) capture the direction-dependant way that sound interacts with the head and torso. In virtual audio systems, which aim to emulate these effects, non-individualized, generic HRTFs are typically used leading to an inaccurate perception of virtual sound location. Training has the potential to exploit the brain’s ability to adapt to these unfamiliar cues. In this study, three virtual sound localization training paradigms were evaluated; one provided simple visual positional confirmation of sound source location, a second introduced game design elements (“gamification”) and a final version additionally utilized head-tracking to provide listeners with experience of relative sound source motion (“active listening”). The results demonstrate a significant effect of training after a small number of short (12-minute) training sessions, which is retained across multiple days. Gamification alone had no significant effect on the efficacy of the training, but active listening resulted in a significantly greater improvements in localization accuracy. In general, improvements in virtual sound localization following training generalized to a second set of non-individualized HRTFs, although some HRTF-specific changes were observed in polar angle judgement for the active listening group. The implications of this on the putative mechanisms of the adaptation process are discussed.

Reaching to sounds in virtual reality: A multisensory-motor approach to re-learn sound localisation

“…Finally, following previous works (Rabini et al, 2019;Steadman et al, 2019;Valzolgher, Campus, Rabini, Gori & Pavani, under review) we examined whether re-learning across blocks relates to the magnitude of the immediate ear-plug effect. To this aim, we correlated across participants the ear-plug effect (measured as the difference in absolute localisation error between B and M1; positive values indicate larger plug-effect) and re-learning (measured as the difference in absolute localisation error between M1 and M4; positive values indicate larger re-learning).…”

“…While re-learning of sound-space correspondences has been previously documented (Carlile, 2014;Kacelnik, Nodal, Parsons & King, 2006;Keating & King, 2015;Nawaz, McNeill & Greenberg, 2014;Steadman et., 2019), to the best of our knowledge, fast improvement over trials have not been studied and described before. Traditionally, updating of space-sound correspondences has been examined in paradigms which comprised pre-and post-testing phases interspersed with training procedures which could occur over several days (e.g., 5 days in Strelnikov et al, 2011 andin Irving &Moore, 2011;10 days in Ohuchi et al, 2005).…”

“…Whenever the other sensory systems (e.g., vision) provide reliable spatial information about auditory events, the brain exploits these additional sensory sources to calibrate and optimize internal models for spatial hearing (Keating & King, 2015). This notion emerged, for instance, from studies that examined re-learning of sound-space correspondences when auditory cues were temporarily altered using monaural ear-plugs (Rabini et al, 2019;Strelninkov et al, 2011;Trapeau & Schönwiesner, 2015), ear molds (Van Wanrooij & Van Opstal, 2005) or non-individualised HRTFs (Head-Related Transfer Functions; Honda, Shibata, Gyoba, Saitou, Iwaya & Suzuki, 2007;Parseihian & Katz, 2012;Steadman, Kim, Lestang, Goodman & Picinali, 2019). In these simulated altered-listening conditions, multisensory training procedures proved effective for re-learning sound-space correspondences (for reviews see: Carlile, 2014;Keating & King, 2015;Knudsen & Knudsen, 1985;Mendonça, 2014;Irving & Moore, 2011).…”

“…This training improved vertical localisation performances and reduced front/back confusion errors. A more recent study (Steadman et al, 2019) combined gamification, virtual reality and active listening (specifically, free headmovements) in a training protocol aimed at ameliorant sound localisation with virtual sounds. These authors 6 trained participants to adapt to non-individualized HRTFs using virtual reality.…”

Short-term effects of sound localization training in virtual reality

“…The Unity project used to build the software for the iPhone 6 and the full dataset, including Matlab scripts to read it, are publicly available on Zenodo 60 . Please note that the authors do not have permission to redistribute the LIMSI Spatialization Engine, which is required to reproduce the stimuli.…”