Speech understanding oppositely affects acoustic and linguistic neural tracking in a speech rate manipulation paradigm

Abstract

When listening to continuous speech, the human brain can track features of the presented speech signal. It has been shown that neural tracking of acoustic features is a prerequisite for speech understanding and can predict speech understanding in controlled circumstances. However, the brain also tracks linguistic features of speech, which may be more directly related to speech understanding. We investigated acoustic and linguistic speech processing as a function of varying speech understanding by manipulating the speech rate. In this paradigm, acoustic and linguistic speech processing are affected simultaneously but in opposite directions: When the speech rate increases, more acoustic information per second is present. In contrast, linguistic information decreases as speech becomes less intelligible at higher speech rates. We measured the EEG of 18 participants who listened to speech at various speech rates. As expected and confirmed by the behavioral results, speech understanding decreased with increasing speech rate. Accordingly, linguistic neural tracking decreased with increasing speech rate, but acoustic neural tracking increased. This indicates that neural tracking of linguistic representations can capture the gradual effect of decreasing speech understanding. In addition, increased acoustic neural tracking does not necessarily imply better speech understanding. This suggests that, although more challenging to measure due to the low signal-to-noise ratio, linguistic neural tracking may be a more direct predictor of speech understanding.

Heard or understood? Neural tracking of language features in a comprehensible story, an incomprehensible story and a word list

“…However, as EEG follows a 1/f-characteristics, a broader frequency range would follow similar patterns as the delta band. In line with our results, Kosem et al (2022) and Verschueren et al (2022) also did not see a link between acoustic tracking and speech comprehension. Altogether, this suggests that acoustic tracking is a necessity rather than a sufficient condition for speech comprehension.…”

“…This suggests that there is a link between acoustic tracking and speech comprehension. However, recent studies by Kosem et al (2022) and Verschueren et al (2022) showed that this link between acoustic tracking and comprehension is not always observed. More specifically, by manipulating speech comprehension by either using noise-vocoded speech or changing the speech rate, they observed that as the speech became comprehensible, acoustic tracking did not increase.…”

A tradeoff between acoustic and linguistic feature encoding in spoken language comprehension

“…If this is so, then language comprehension can be seen as a neural filter over sensory and perceptual input in service of the transformation of that input into linguistic structures. A similar effect was also shown for increasing speech rate; tracking performance of linguistic features decreased and tracking performance of acoustic features increased with decreasing intelligibility (Verschueren et al, 2022).…”

Explaining flexible continuous speech comprehension from individual motor rhythms

“…Interestingly, however, in both experiments performance decreased later than previously observed, that is, beyond rates of 9 syllables/s(56,80). However, in line with our findings, several other studies, also observed shallower decreases in speech comprehension, with relatively high comprehension at higher syllable rates (~12 syllables/s)(3,56,81,82). We consider several possible explanations for these discrepancies.…”