Reduction of the Harmonic Series Influences Musical Enjoyment With Cochlear Implants

Abstract

Objective Cochlear implantation is associated with poor music perception and enjoyment. Reducing music complexity has been shown to enhance music enjoyment in cochlear implant (CI) recipients. In this study, we assess the impact of harmonic series reduction on music enjoyment. Study Design Prospective analysis of music enjoyment in normal-hearing (NH) individuals and CI recipients. Setting Single tertiary academic medical center. Patients NH adults (N=20) and CI users (N=8) rated the Happy Birthday song on three validated enjoyment modalities–musicality, pleasantness, and naturalness. Intervention Subjective rating of music excerpts. Main outcome measures Participants listened to seven different instruments play the melody, each with five levels of harmonic reduction (Full|F3+F2+F1+F0|F2+F1+F0|F1+F0|F0). NH participants listened to the segments both with and without CI simulation. Linear mixed effect models (LME) and likelihood ratio tests were used to assess the impact of harmonic reduction on enjoyment. Results NH listeners without simulation rated segments with the first four harmonics (F3+F2+F1+F0) most pleasant and natural (p<0.001|p=0.004). NH listeners with simulation rated the first harmonic alone (F0) most pleasant and natural (p<0.001|p=0.003). Their ratings demonstrated a positive linear relationship between harmonic reduction and both pleasantness (slope estimate=0.030|SE=0.004|p<0.001|LME) and naturalness (slope estimate=0.012|SE=0.003|p=0.003|LME). CI recipients also found the first harmonic alone (F0) to be most pleasant (p=0.003), with a positive linear relationship between harmonic reduction and pleasantness (slope estimate=0.029|SE=0.008|p<0.001|LME). Conclusions Harmonic series reduction increases music enjoyment in CI and NH individuals with or without CI simulation. Therefore, minimization of the harmonics may be a useful strategy for enhancing musical enjoyment among both NH and CI listeners.

Interactive Evaluation of a Music Preprocessing Scheme for Cochlear Implants Based on Spectral Complexity Reduction

“…The processed signals from the chamber music database used in the listening experiments exhibit a lower spectral complexity as both the harmonic components of the leading voice and the accompanying voices are attenuated. Hence the preference for a high spectral complexity reduction found in the participating CI users is in line with the findings by Nemer et al (2017) where CI users reported an increase in enjoyment of monophonic pieces with a reduced overtone series, and the findings by Kohlberg et al (2015) where CI users rated recordings of pieces with a reduced number of competing instruments as more pleasant than the original. But in contrast to the method used by Nemer et al, the PCA-based spectral complexity reduction scheme does not reduce the overtone series in an ordered fashion from higher to lower order toward the fundamental frequency.…”

“…As major changes in electrode design are not to be expected in the short run, we consider signal preprocessing techniques, besides additional rehabilitation and training efforts, to be the major means in facilitating music appraisal in CI users. The found preference for signals with a reduced number of retained PCA components is in line with the evaluation of other preprocessing methods where the spectral complexity was reduced by reducing the number of overtones manually (Nemer et al, 2017) or by remixing music signals to enhance the leading voices, vocals and rhythmic components (Kohlberg et al, 2015, Pons et al, 2016, Buyens et al, 2017, Gajęcki and Nogueira, 2018). The evaluated method in the study at hand directly tackles the harmonic and pitch-related features of musical signals to reduce the impact of spectral spread in CI listeners.…”

“…Similarly, Kohlberg et al (2015) produced remixes of a multi-track recording of a country music song containing one to five of originally ten instruments and found a significant preference for remixes containing only one to three instruments as compared to the original music pieces in a listening experiment with CI listeners. Moreover, Nemer et al (2017) showed that reducing the number of overtones in a monophonic melody actually helps CI listeners to perceive music as more pleasant. In contrast to those methods approaches have been proposed that do not rely on multi-track recordings or manual preprocessing: Based on their earlier work the authors of Buyens et al (2015) proposed source separation and remixing schemes for pop and rock music using harmonic/percussive sound separation (HPSS) to accentuate primarily vocals and drums with respect to other instruments and evaluated them in listening experiments with CI listeners (Buyens et al, 2017).…”

The Perception of Multiple Simultaneous Pitches as a Function of Number of Spectral Channels and Spectral Spread in a Noise-Excited Envelope Vocoder

“…More importantly, mixtures of concurrent complex tones (such as multiple instruments played simultaneously) lead to a drastic decrease in peripheral resolvability of the resulting mixture, even if all the component complexes consist of low-numbered resolved harmonics (e.g., Micheyl et al 2010b;Graves and Oxenham 2019). Studies have shown that judgments of music quality in both CI users and hearing-impaired listeners decrease when multiple instruments are involved (Looi et al 2007;Nemer et al 2017). Hence, it is important to investigate whether the resolution deemed sufficient for single tones is the same or higher for mixtures of complex tones.…”

A model for music complexity applied to music preprocessing for cochlear implants

“…In [6] a different approach to reduce (spectral) complexity was investigated in order to increase melody clarity and ease of listening for CI users in instrumental music with reduced-rank approximations of music signals. In [7] the impact of harmonic series reduction on music enjoyment was investigated with NH and CI subjects.…”