Sad and happy emotion discrimination in music by children with cochlear implants

Abstract

Children using cochlear implants (CIs) develop speech perception but have difficulty perceiving complex acoustic signals. Mode and tempo are the two components used to recognize emotion in music. Based on CI limitations, we hypothesized children using CIs would have impaired perception of mode cues relative to their normal hearing peers and would rely more heavily on tempo cues to distinguish happy from sad music. Study participants were children with 13 right CIs and 3 left CIs (M = 12.7, SD = 2.6 years) and 16 normal hearing peers. Participants judged 96 brief piano excerpts from the classical genre as happy or sad in a forced-choice task. Music was randomly presented with alterations of transposed mode, tempo, or both. When music was presented in original form, children using CIs discriminated between happy and sad music with accuracy well above chance levels (87.5%) but significantly below those with normal hearing (98%). The CI group primarily used tempo cues, whereas normal hearing children relied more on mode cues. Transposing both mode and tempo cues in the same musical excerpt obliterated cues to emotion for both groups. Children using CIs showed significantly slower response times across all conditions. Children using CIs use tempo cues to discriminate happy versus sad music reflecting a very different hearing strategy than their normal hearing peers. Slower reaction times by children using CIs indicate that they found the task more difficult and support the possibility that they require different strategies to process emotion in music than normal.

Experience Changes How Emotion in Music Is Judged: Evidence from Children Listening with Bilateral Cochlear Implants, Bimodal Devices, and Normal Hearing

“…As shown in Fig 3A , children using CIs identified emotions conveyed by original melodies with high accuracy (mean ± SE, 86.3 ± 1.4%) but did not score as high as their peers with normal hearing (94.3 ± 1.8%). Data in both groups were similar to the cohorts in our previous study [ 75 ]. Children with CIs reacted differently from their normal hearing peers when cues were changed ( Fig 3B ).…”

“…However, poor spectral representation provided by CIs proved highly unsatisfactory for music which, being less redundant than language, requires a much more precise and detailed processing of the spectral aspects of the stimulus. It is clear that children using unilateral CIs hear rhythm better than tonal aspects of music [ 73 ] and use tempo cues to judge the emotion in music [ 75 ], but these skills could be different in children using bilateral CIs or in children who have considerable residual hearing and use a CI combined with acoustic hearing in the other ear (bimodal users). As shown in Fig 3A , children using CIs identified emotions conveyed by original melodies with high accuracy (mean ± SE, 86.3 ± 1.4%) but did not score as high as their peers with normal hearing (94.3 ± 1.8%).…”

“…Thus, given the technical constraints of signal transduction, which places a greater importance to the rate of stimulation rather than a faithful reproduction of the envelope of the waveforms, unilateral CI users rely more on tempo than mode cues because of the signal from the CI device itself [ 73 , 75 ]. If this holds true for children using unilateral CIs, it may or may not be true of children using bilateral CIs.…”

Vocal Emotion Identification by Children Using Cochlear Implants, Relations to Voice Quality, and Musical Interests

“…In particular, perception of spectral information and the duration of the phonemes have been shown to be inadequate in people who have HI and CIs (Hopyan-Misakyan, Gordon, Dennis & Papsin, 2009;Jiam, Caldwell, Deroche, Chatterjee & Limb, in 5 press; Luo et al, 2007;Stiles & Nadler, 2013;Wei, Cao, & Zeng, 2004). Listeners with HI seem to rely more on tempo, speech rate and its variations, and intensity in emotion perception (Giannantonio, Polonenko, Papsin, Paludetti & Gordon, 2015;Hopyan, Manno III, Papsin & Gordon, 2016;Jiam et al, in press;Nakata, Trehub & Kanda, 2012).…”

Musical Emotion Perception in Bimodal Patients: Relative Weighting of Musical Mode and Tempo Cues

“…Specifically, CI users' ratings of stimuli with the same tempo were similar, irrespective of mode, while normal-hearing listener ratings' differed significantly for varying mode. Similarly, Hopyan et al (2016) altered mode, tempo, or both mode and tempo, and found that CI users relied predominantly on tempo. These findings are consistent with previous literature demonstrating that spectral cues are poorly represented for CI users, whereas temporal cues remain robust.…”