Speech Recognition with Primarily Temporal Cues

Abstract

Nearly perfect speech recognition was observed under conditions of greatly reduced spectral information. Temporal envelopes of speech were extracted from broad frequency bands and were used to modulate noises of the same bandwidths. This manipulation preserved temporal envelope cues in each band but restricted the listener to severely degraded information on the distribution of spectral energy. The identification of consonants, vowels, and words in simple sentences improved markedly as the number of bands increased; high speech recognition performance was obtained with only three bands of modulated noise. Thus, the presentation of a dynamic temporal pattern in only a few broad spectral regions is sufficient for the recognition of speech.

Level-Dependent Changes in Perception of Speech Envelope Cues

“…Although most studies of speech recognition focus on information contained in speech energy in the spectral domain, substantial evidence confirms that amplitude modulations in the speech signal carry information important for communication (Rosen 1992;Shannon et al 1995). The speech signal contains varying degrees of envelope fluctuation, which convey different types of information.…”

“…In some studies, near-normal recognition of speech that contains little spectral information has been achieved when the speech temporal envelope is preserved in a relatively small number of frequency channels. Indeed, Shannon et al (1995) reported that four-channel "vocoded" speech, which had very limited frequency information but relatively intact temporal envelope information, was highly intelligible.…”

Auditory sensory deficits in developmental dyslexia: A longitudinal ERP study

“…Selectively degrading modulation frequencies near the syllable rate (4-16 Hz) degrades participants' ability to identify consonants and to understand sentences (Drullman et al, 1994). In contrast, speech stimuli that are processed to leave only relatively slow (below 40 Hz) temporal modulations enable near-perfect speech intelligibility (Shannon et al, 1995). When the amplitude envelope is analysed in terms of its constituent temporal modulation frequencies, the dominant modulation frequencies are around 4-6 Hz, reflecting the sequential rate of words and syllables (Drullman, 2006).…”

Lexical Information Drives Perceptual Learning of Distorted Speech: Evidence From the Comprehension of Noise-Vocoded Sentences.

“…We use noise-vocoded speech, which is an acoustic distortion that preserves temporal information while removing the temporal fine structure and spectral detail of speech (Shannon et al, 1995). Although initially unintelligible, noisevocoded sentences can be readily understood following a period of training.…”