Feasibility of vocal emotion conversion on modulation spectrogram for simulated cochlear implants

Abstract

Abstract-Cochlear implant (CI) listeners were found to have great difficulty with vocal emotion recognition because of the limited spectral cues provided by CI devices. Previous studies have shown that the modulation spectral features of temporal envelopes may be important cues for vocal emotion recognition of noise-vocoded speech (NVS) as simulated CIs. In this paper, the feasibility of vocal emotion conversion on a modulation spectrogram for simulated CIs for correctly recognizing vocal emotion is confirmed. A method based on a linear prediction scheme is proposed to modify the modulation spectrogram and its features of neutral speech to match that of emotional speech. The logic of this approach is that if vocal emotion perception of NVS is based on the modulation spectral features, NVS with similar modulation spectral features of emotional speech will be recognized as the same emotion. As a result, it was found that the modulation spectrogram of neutral speech can be successfully converted to that of emotional speech. The results of the evaluation experiment showed the feasibility of vocal emotion conversion on the modulation spectrogram for simulated CIs. The vocal emotion enhancement on the modulation spectrogram was also further discussed.

Contributions of Temporal Modulation Cues in Temporal Amplitude Envelope of Speech to Urgency Perception

“…These studies found that significant cues for perception of non-linguistic information are temporal modulation components of the noise-vocoded stimuli lower than the modulation frequency of 8 Hz. These studies also suggested that recognition of vocal-emotion and speaker individuality using the NVS scheme with NH listeners can be used to predict the response from CI listeners as CI simulators [24][25][26].…”