Voice emotion recognition by cochlear-implanted children and their normally-hearing peers

Abstract

Despite their remarkable success in bringing spoken language to hearing impaired listeners, the signal transmitted through cochlear implants (CIs) remains impoverished in spectro-temporal fine structure. As a consequence, pitch-dominant information such as voice emotion, is diminished. For young children, the ability to correctly identify the mood/intent of the speaker (which may not always be visible in their facial expression) is an important aspect of social and linguistic development. Previous work in the field has shown that children with cochlear implants (cCI) have significant deficits in voice emotion recognition relative to their normally hearing peers (cNH). Here, we report on voice emotion recognition by a cohort of 36 school-aged cCI. Additionally, we provide for the first time, a comparison of their performance to that of cNH and NH adults (aNH) listening to CI simulations of the same stimuli. We also provide comparisons to the performance of adult listeners with CIs (aCI), most of whom learned language primarily through normal acoustic hearing. Results indicate that, despite strong variability, on average, cCI perform similarly to their adult counterparts; that both groups’ mean performance is similar to aNHs’ performance with 8-channel noise-vocoded speech; that cNH achieve excellent scores in voice emotion recognition with full-spectrum speech, but on average, show significantly poorer scores than aNH with 8-channel noise-vocoded speech. A strong developmental effect was observed in the cNH with noise-vocoded speech in this task. These results point to the considerable benefit obtained by cochlear-implanted children from their devices, but also underscore the need for further research and development in this important and neglected area.

Contributions of temporal cue on the perception of speaker individuality and vocal emotion for noise-vocoded speech

“…Moreover, child CI listeners succeeded in differentiating their mothers' utterances from those of other people [22]. For vocal-emotion recognition, Chatterjee et al [19] found that the mean performance of CI listeners was similar to that of NH listeners with 8-band NVS which was much better than the chance level. The temporal envelope of speech has been suggested to potentially be an important cue in the perception of nonlinguistic information.…”

“…It has been known that CI listeners' performances of vocal-emotion and speaker recognition are poorer than normal-hearing (NH) listeners, as the poor spectral cues provided by CI device [17][18][19][20]. Nonetheless, for speaker recognition, Krull et al showed that training results in improved recognition rates of speaker in CI simulations [21].…”

Recognizing the journey and celebrating the achievement of cochlear implants

“…Pfingst et al, 2015 present human and animal studies on not only the importance of cochlear health for implant function but also tissue-engineering procedures for improving or even replacing the implant function. Other researchers present various approaches to further improve cochlear implant function and expand its utilities, including reducing or exploiting channel interaction (Kalkman et al, 2015;Schatzer et al, 2015); coordinated electro-acoustic stimulation (Dorman et al, 2015;Tillein et al, 2015); bilateral implantation (Kan and Litovsky, 2015;Laback et al, 2015); learning or training (Chatterjee et al, 2015;Svirsky et al, 2015;van Wieringen and Wouters, 2015); audio-visual integration and brain imaging (Strelnikova et al, 2015); and lowering the cost (Zeng et al, 2015).…”

Toddlers' comprehension of degraded signals: Noise-vocoded versus sine-wave analogs

“…Thus the eight-channel NV stimuli used in this study might be more representative of the type of spectral degradation children might experience. In a recent study of voice emotion recognition, Chatterjee et al (2015) reported that school-aged children with CIs attending to full-spectrum speech showed mean performance similar to hearing adults with eight-channel NV speech. In contrast, the performance of hearing children with eight-channel NV speech was significantly poorer than that of hearing adults and showed a strong developmental effect.…”