Assessment of a visual to spatial-audio sensory substitution system

Mhaish, Ahmad; Gholamalizadeh, Torkan; İnce, Gökhan; Duff, Damien Jade

doi:10.1109/siu.2016.7495723

Cited by 1 publication

(2 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then a synthesized stereophonic signal simulating the distortions occurring while receiving the audio signal by the two ears is obtained. Among the SSDs used in localization experiments, the Synaestheatre (Hamilton-Fletcher et al, 2016a ; Richardson et al, 2019 ), the Vibe (Hanneton et al, 2010 ) and the one presented by Mhaish et al ( 2016 ) spatialize azimuth (lateral position) and elevation (vertical position). Other SSDs only spatialize the azimuth: the See differently device (Rouat et al, 2014 ), the one studied in Ambard et al ( 2015 ), and the recent one presented in Scalvini et al ( 2022 ).…”

Section: Introductionmentioning

confidence: 99%

“…Many studies investigating static object localization abilities have already been conducted with blindfolded sighted persons using visual-to-auditory SSDs. Various types of tasks have already been used, for example discrimination tasks with forced choice (Proulx et al, 2008 ; Levy-Tzedek et al, 2012 ; Ambard et al, 2015 ; Mhaish et al, 2016 ; Richardson et al, 2019 ), grasping tasks (Proulx et al, 2008 ), index or tool pointing tasks (Auvray et al, 2007 ; Hanneton et al, 2010 ; Brown et al, 2011 ; Pourghaemi et al, 2018 ; Commère et al, 2020 ), or head-pointing tasks (Scalvini et al, 2022 ). Those studies showed the high potential of SSDs to localize an object and interact with it.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Cross-modal correspondence enhances elevation localization in visual-to-auditory sensory substitution

et al. 2023

View full text Add to dashboard Cite

IntroductionVisual-to-auditory sensory substitution devices are assistive devices for the blind that convert visual images into auditory images (or soundscapes) by mapping visual features with acoustic cues. To convey spatial information with sounds, several sensory substitution devices use a Virtual Acoustic Space (VAS) using Head Related Transfer Functions (HRTFs) to synthesize natural acoustic cues used for sound localization. However, the perception of the elevation is known to be inaccurate with generic spatialization since it is based on notches in the audio spectrum that are specific to each individual. Another method used to convey elevation information is based on the audiovisual cross-modal correspondence between pitch and visual elevation. The main drawback of this second method is caused by the limitation of the ability to perceive elevation through HRTFs due to the spectral narrowband of the sounds.MethodIn this study we compared the early ability to localize objects with a visual-to-auditory sensory substitution device where elevation is either conveyed using a spatialization-based only method (Noise encoding) or using pitch-based methods with different spectral complexities (Monotonic and Harmonic encodings). Thirty eight blindfolded participants had to localize a virtual target using soundscapes before and after having been familiarized with the visual-to-auditory encodings.ResultsParticipants were more accurate to localize elevation with pitch-based encodings than with the spatialization-based only method. Only slight differences in azimuth localization performance were found between the encodings.DiscussionThis study suggests the intuitiveness of a pitch-based encoding with a facilitation effect of the cross-modal correspondence when a non-individualized sound spatialization is used.

show abstract