SPHERE: A novel approach to 3D and active sound localization

Abstract

In everyday life, localizing a sound source in free-field entails more than the sole extraction of monaural and binaural auditory cues to define its location in the three-dimensions (azimuth, elevation and distance). In spatial hearing, we also take into account all the available visual information (e.g., cues to sound position, cues to the structure of the environment), and we resolve perceptual ambiguities through active listening behavior, exploring the auditory environment with head or/and body movements. Here we introduce a novel approach to sound localization in 3D named SPHERE (European patent n° WO2017203028A1), which exploits a commercially available Virtual Reality Head-mounted display system with real-time kinematic tracking to combine all of these elements (controlled positioning of a real sound source and recording of participants' responses in 3D, controlled visual stimulations and active listening behavior). We prove that SPHERE allows accurate sampling of the 3D spatial hearing abilities of normal hearing adults, and it allowed detecting and quantifying the contribution of active listening. Specifically, comparing static vs. free head-motion during sound emission we found an improvement of sound localization accuracy and precisions. By combining visual virtual reality, real-time kinematic tracking and real-sound delivery we have achieved a novel approach to the study of spatial hearing, with the potentials to capture real-life behaviors in laboratory conditions. Furthermore, our new approach also paves the way for clinical and industrial applications that will leverage the full potentials of active listening and multisensory stimulation intrinsic to the SPHERE approach for the purpose rehabilitation and product assessment.

Adapting to altered auditory cues: Generalization from manual reaching to head pointing

“…The software is designed to guide the experimenter to align the real loudspeaker (i.e., the sound source) with a set of pre-determined positions defined in the 3D virtual environment in each trial. This method combining virtual reality and kinematic tracking to measure sound localization abilities has been developed in our laboratory [ 37 ] and has been already adopted in previous studies [ 25 , 38 ].…”

Reaching to Sounds Improves Spatial Hearing in Bilateral Cochlear Implant Users

“…This task was carried out entirely in VR, always using real sounds delivered in free field from predetermined positions computed on each trial based on the initial head position (Coudert et al 2022;Valzolgher et al 2020a,c). Our VR approach brings real sounds into visual VR and coordinates everything using Unity (see the Materials in Supplemental Digital Content 2, http://links.lww.com/EANDH/B43 for details about the apparatus and also Gaveau et al 2020). Participants were immersed in a virtual room that matched the size of the real one but was devoid of any objects.…”

“…When the correct posture was reached, the fixation cross turned from white to blue. In the meanwhile, the experimenter placed the speaker in one of the possible eight predetermined positions, using visual indications provided on a dedicated monitor (see Verdelet et al 2019 andGaveau et al 2020 for a validation of this method and Valzolgher et al 2020a,c for examples). The predetermined positions (shown in the top part of Fig.…”