Crossmodal interactions during non-linguistic auditory processing in cochlear-implanted deaf patients

Abstract

Due to signal distortion, speech comprehension in cochlear-implanted (CI) patients relies strongly on visual information, a compensatory strategy supported by important cortical crossmodal reorganisations. Though crossmodal interactions are evident for speech processing, it is unclear whether a visual influence is observed in CI patients during non-linguistic visual-auditory processing, such as face-voice interactions, which are important in social communication. We analyse and compare visual-auditory interactions in CI patients and normal-hearing subjects (NHS) at equivalent auditory performance levels. Proficient CI patients and NHS performed a voice-gender categorisation in the visual-auditory modality from a morphing-generated voice continuum between male and female speakers, while ignoring the presentation of a male or female visual face. Our data show that during the face-voice interaction, CI deaf patients are strongly influenced by visual information when performing an auditory gender categorisation task, in spite of maximum recovery of auditory speech. No such effect is observed in NHS, even in situations of CI simulation. Our hypothesis is that the functional crossmodal reorganisation that occurs in deafness could influence nonverbal processing, such as face-voice interaction; this is important for patient internal supramodal representation.

Rapid Assessment of Non-Verbal Auditory Perception in Normal-Hearing Participants and Cochlear Implant Users

“…The participants were not diagnosed with psychiatric or neurological disorders, as assessed by a short questionnaire at inclusion in the study. The control participants were selected for their hearing-in-quiet capacities rather than their age, leading to age differences between the groups [ 20 , 69 , 90 ]. As age-difference could have played a role in the CI results, we performed Pearson correlations between each score obtained in the Results section and the age of the CI users, with Bonferroni correction for multiple testing.…”

“…This audiovisual enhancement is more efficient in CI users compared to NH participants for all types of speech, even foreign-accented speech [ 68 ]. One study on non-linguistic speech processing, using voice gender identification, suggested that CI users are influenced more strongly by visual information than are NH participants, even for tasks not directly related to speech comprehension [ 69 ].…”

Recognizing voices through a cochlear implant: A systematic review

“…The vast majority of these tokens were produced naturally and used as stimuli in their natural form (n = 29), while other studies used synthetic or synthesized speech (n = 8). For example, Barone et al (2016) recorded two voices (one female and one male) producing a word, and using a specialized speech manipulation software, they morphed these two voices to a continuum of 11 voices, ranging from 100% female to 100% male and 9 gender-interpolated voices in between.…”

“…Other studies found significant differences between groups in the manner by which listeners encoded vocal properties (see Section 3.5; Barone et al, 2016;Kovačić & Balaban, 2009;Meister et al, 2016;Zaltz et al, 2018). Another study examined whether topdown information, such as visual information, would influence listeners' gender rating of voices (Barone et al, 2016).…”

“…Specifically, CI users had poor performance when they were tested on the first month of CI use, but improved over time. Only one study specifically looked at the effects of hearing experience on voice perception in older adults, and they found no relation between factors such as age, duration of deafness or duration of CI experience and participants' performance on a gender identification task (Barone et al, 2016).…”