Electro-haptic stimulation enhances speech recognition in spatially separated noise for cochlear implant users

Abstract

Hundreds of thousands of profoundly hearing-impaired people perceive sounds through electrical stimulation of the auditory nerve using a cochlear implant (CI). However, CI users are often poor at understanding speech in noisy environments and separating sounds that come from different locations. We provided missing speech and spatial hearing cues through haptic stimulation to augment the electrical CI signal. After just 30 min of training, we found this “electro-haptic” stimulation substantially improved speech recognition in multi-talker noise when the speech and noise came from different locations. Our haptic stimulus was delivered to the wrists at an intensity that can be produced by a compact, low-cost, wearable device. These findings represent a significant step towards the production of a non-invasive neuroprosthetic that can improve CI users’ ability to understand speech in realistic noisy environments.

Sensitivity to haptic sound-localisation cues

“…However, the fingertips were not considered a suitable site for a haptic device as they are used in numerous everyday tasks. Instead, this study focused on the wrist as wrist-worn devices are commonplace and so are aesthetically acceptable, do not impede everyday tasks, and have been used in previous studies for enhancing listening using haptics 1 , 2 , 4 , 6 .…”

“…After only minimal training using this approach, CI users had dramatically improved sound-localisation accuracy, which was similar to that of bilateral hearing-aid users 1 , 6 . Using the same approach, haptic stimulation was found to improve speech-reception thresholds in spatially-separated noise by around 3 dB 2 .…”

“…This “electro-haptic” stimulation 4 has been shown to substantially improve sound localisation 1 , 6 and segregation 2 , 4 , 7 , as well as discrimination of more basic sound properties such as pitch 8 . In studies showing benefits to sound localisation and segregation, haptic signals have been extracted from the audio received by hearing-assistive devices behind each ear and delivered to each corresponding wrist 1 , 2 , 6 . After only minimal training using this approach, CI users had dramatically improved sound-localisation accuracy, which was similar to that of bilateral hearing-aid users 1 , 6 .…”

“…The dominant spatial hearing cues are time and intensity differences across the ears (interaural time and intensity differences). In previous electro-haptic stimulation studies, both interaural time (ITD) and intensity (IID) difference cues were delivered to the wrists 1 , 2 , 6 . However, it was unclear whether one or both cues were used to locate and separate sounds.…”

“…The second aim was to compare sensitivity for signals both with and without amplitude modulation applied. This is important, as studies showing that haptic stimulation on the wrists can improve speech-in-noise and sound-localisation performance in CI users have modulated the amplitude of the haptic signal using the speech amplitude envelope 1 , 2 , 4 , 6 , 7 . The tactile system is known to be highly sensitive to amplitude envelope differences 9 .…”