Spectral and temporal cues for speech recognition: Implications for auditory prostheses

Abstract

Features of stimulation important for speech recognition in people with normal hearing and in people using implanted auditory prostheses include spectral information represented by place of stimulation along the tonotopic axis and temporal information represented in low-frequency envelopes of the signal. The relative contributions of these features to speech recognition and their interactions have been studied using vocoder-like simulations of cochlear implant speech processors presented to listeners with normal hearing. In these studies, spectral/place information was manipulated by varying the number of channels and the temporal-envelope information was manipulated by varying the lowpass cutoffs of the envelope extractors. Consonant and vowel recognition in quiet reached plateau at 8 and 12 channels and lowpass cutoff frequencies of 16 Hz and 4 Hz, respectively. Phoneme (especially vowel) recognition in noise required larger numbers of channels. Lexical tone recognition required larger numbers of channels and higher lowpass cutoff frequencies. There was a tradeoff between spectral/place and temporal-envelope requirements. Most current auditory prostheses seem to deliver adequate temporal-envelope information but the number of effective channels is suboptimal, particularly for speech recognition in noise, lexical tone recognition and music perception.

Contributions of temporal cue on the perception of speaker individuality and vocal emotion for noise-vocoded speech

“…Xu and Pfingst measured both consonant and vowel recognition as a function of the number of channels (1 to 16) and upper limit of modulation frequency (1 to 512 Hz) [4]. The knee points of vowel recognition for different numbers of channels are all below about 4 Hz.…”

“…The signal in each band can be regarded as a temporal amplitude envelope with a carrier (temporal fine structure). The temporal envelope of speech has been proved to be an important cue for speech perception from the studies using noise-vocoded speech (NVS) [1][2][3][4]. NVS is generated by replacing the carriers with band-limited noise, so the spectral cue is reduced dramatically and the temporal cue is preserved.…”

“…Experiments of speaker and vocal-emotion recognition were carried out using an analysis/synthesis method of noise-vocoded speech (NVS). The temporal resolution of NVS was controlled by varying the upper limit of modulation frequency (0, 0.5,1,2,4,8,16, 32, and 64 Hz). In addition, the role of temporal cue in the different spectral resolution condition was also investigated by varying the number of channels (4, 8, and 16).…”

Band importance functions of listeners with cochlear implants using clinical maps

“…This is surprising, because listeners with normal hearing did not show much importance for low frequency channels. This increased reliance on low frequency channels may be due to the fact that cochlear implants have relatively poor ability to convey place of excitation cues, but are better at conveying temporal envelope cues (Xu and Pfingst, 2008). Experiments in listeners with normal hearing with transposed tones, sounds where low rate modulation is applied to a high frequency carrier, demonstrated that correct (i.e., low frequency) place of excitation is necessary to perceive the fundamental frequency of low rate modulation (Oxenham et al, 2004).…”

Consonant recognition as a function of the number of stimulation channels in the Hybrid short-electrode cochlear implant

“…For more difficult speech with multiple talkers, and for speech in noise, more channels are needed; long-electrode CI users continue to improve their reiss@ohsu.edu speech recognition up to 7-8 channels (Friesen et al, 2001). In contrast, NH subjects listening to a simulation of longelectrode processed speech improve their speech recognition up to 22 or more channels and so achieve higher speech recognition scores than CI subjects (Friesen et al, 2001;Xu and Pfingst, 2008). One proposed explanation for the limited ability of CI users to improve speech recognition with more than 7-8 stimulation channels out of up to 22 available intracochlear electrodes is that current spread and the resulting channel interactions limit the speech recognition benefit to every third electrode.…”