Predictors of Emotional Prosody Identification by School-Age Children With Cochlear Implants and Their Peers With Normal Hearing

Abstract

Objectives: Children with cochlear implants (CIs) vary widely in their ability to identify emotions in speech. The causes of this variability are unknown, but this knowledge will be crucial if we are to design improvements in technological or rehabilitative interventions that are effective for individual patients. The objective of this study was to investigate how well factors such as age at implantation, duration of device experience (hearing age), nonverbal cognition, vocabulary, and socioeconomic status predict prosody-based emotion identification in children with CIs, and how the key predictors in this population compare to children with normal hearing who are listening to either normal emotional speech or to degraded speech. Design: We measured vocal emotion identification in 47 school-age CI recipients aged 7 to 19 years in a single-interval, 5-alternative forced-choice task. None of the participants had usable residual hearing based on parent/caregiver report. Stimuli consisted of a set of semantically emotion-neutral sentences that were recorded by 4 talkers in child-directed and adult-directed prosody corresponding to five emotions: neutral, angry, happy, sad, and scared. Twenty-one children with normal hearing were also tested in the same tasks; they listened to both original speech and to versions that had been noise-vocoded to simulate CI information processing. Results: Group comparison confirmed the expected deficit in CI participants’ emotion identification relative to participants with normal hearing. Within the CI group, increasing hearing age (correlated with developmental age) and nonverbal cognition outcomes predicted emotion recognition scores. Stimulus-related factors such as talker and emotional category also influenced performance and were involved in interactions with hearing age and cognition. Age at implantation was not predictive of emotion identification. Unlike the CI participants, neither cognitive status nor vocabulary predicted outcomes in participants with normal hearing, whether listening to original speech or CI-simulated speech. Age-related improvements in outcomes were similar in the two groups. Participants with normal hearing listening to original speech showed the greatest differences in their scores for different talkers and emotions. Participants with normal hearing listening to CI-simulated speech showed significant deficits compared with their performance with original speech materials, and their scores also showed the least effect of talker- and emotion-based variability. CI participants showed more variation in their scores with different talkers and emotions than participants with normal hearing listening to CI-simulated speech, but less so than participants with normal hearing listening to original speech. Conclusions: Taken together, these results confirm previous findings that pediatric CI recipients have deficits in emotion identification based on prosodic cues, but they improve with age and experience at a rate that is similar to peers with normal hearing. Unlike participants with normal hearing, nonverbal cognition played a significant role in CI listeners’ emotion identification. Specifically, nonverbal cognition predicted the extent to which individual CI users could benefit from some talkers being more expressive of emotions than others, and this effect was greater in CI users who had less experience with their device (or were younger) than CI users who had more experience with their device (or were older). Thus, in young prelingually deaf children with CIs performing an emotional prosody identification task, cognitive resources may be harnessed to a greater degree than in older prelingually deaf children with CIs or than children with normal hearing.

Vocal and musical emotion perception, voice cue discrimination, and quality of life in cochlear implant users with and without acoustic hearing

“…For voice cue perception, we measured voice discrimination in just-noticeable differences (JNDs) for F0 and VTL (Gaudrain & Başkent, 2015, 2018) (Experiment 3b), and for QoL assessment, we used the NCIQ (Hinderink et al, 2000) (Experiment 3c). We hypothesized that (1) if emotion perception mechanisms are similar across materials, vocal and musical emotion categorization outcomes would be correlated, and arousal and valence dimensions would display similar patterns; (2) acoustic hearing would yield higher vocal and musical emotion categorization scores; (3) voice cue discrimination thresholds and QoL scores would correlate with vocal emotion categorization in line with previous findings (e.g., Chatterjee et al, 2023; Von Eiff et al, 2022), and following reports that vocal and musical emotion perception are similar in CI users (Paquette et al, 2018; Volkova et al, 2013), these individual differences would also be reflected in musical emotion categorization.…”

“…When tested for the perception of vocal emotions, CI users typically categorize emotions above chance-level, but display categorization accuracy significantly lower than that of normal-hearing (NH) control participants (Chatterjee et al, 2015(Chatterjee et al, , 2023reviewed in Jiam et al, 2017;Lin et al, 2022;Luo et al, 2007;Paquette et al, 2018;Volkova et al, 2013). Emotion categories are sometimes systematically confused with each other, especially when the categories are located on the same end or half-plane of the arousal dimension, for example happy with scared (e.g., Chatterjee et al, 2015;Lin et al, 2022).…”

“…However, due to the properties of electrode-nerve interface, the transmitted information is reduced in spectrotemporal details (for an overview see Başkent et al, 2016). In particular, the perception of vocal and musical emotion is consistently reported to be challenging for CI users (Ambert-Dahan et al, 2015; Chatterjee et al, 2015, 2023; Lin et al, 2022; Luo et al, 2007; Paquette et al, 2018; Volkova et al, 2013). The ability to perceive vocal emotions in CI users has been shown to be linked to voice cue discrimination thresholds for emotion-related acoustic features such as the voice cues of voice pitch (related to voice fundamental frequency, F0) (Barrett et al, 2020; Deroche et al, 2016; Lin et al, 2022) and vocal tract length (VTL; related to formants, Chuenwattanapranithi et al, 2008; Nussbaum et al, 2022; von Eiff et al, 2022) as well as to listeners’ ratings of quality of life (QoL) (Luo et al, 2018).…”

Sensorineural hearing loss and cognitive impairment: three hypotheses

“…Studies related to the use of hearing aids ( Qian et al, 2016 ; Bucholc et al, 2022 ) and cochlear implants ( Mertens et al, 2021 ) have demonstrated that as hearing aids and cochlear implants help to restore auditory perception, the onset of CI is delayed. It is hypothesized that hearing aids ( Reinten et al, 2021 ) and cochlear implants ( Chatterjee et al, 2023 ) improve the “effortful process” of sound discrimination in the daily lives of SNHL patients, and that this change reduces the previous over-allocation of cognitive resources, helping to restore balance in the CL.…”