Neural markers of speech comprehension: measuring EEG tracking of linguistic speech representations, controlling the speech acoustics

Abstract

When listening to speech, brain responses time-lock to acoustic events in the stimulus. Recent studies have also reported that cortical responses track linguistic representations of speech. However, tracking of these representations is often described without controlling for acoustic properties. Therefore, the response to these linguistic representations might reflect unaccounted acoustic processing rather than language processing. Here we tested several recently proposed linguistic representations, using audiobook speech, while controlling for acoustic and other linguistic representations. Indeed, some of these linguistic representations were not significantly tracked after controlling for acoustic properties. However, phoneme surprisal, cohort entropy, word surprisal and word frequency were significantly tracked over and beyond acoustic properties. Additionally, these linguistic representations are tracked similarly across different stories, spoken by different readers. Together, this suggests that these representations characterize processing of the linguistic content of speech and might allow a behaviour-free evaluation of the speech intelligibility.

Auditory Word Comprehension is Less Incremental in Isolated Words

“…We exclude one additional study on the processing of continuous speech (Di Liberto et al, 2019), which did not report effects of cohort entropy and phoneme surprisal separately. (Brodbeck et al, 2018(Brodbeck et al, , 2021Donhauser & Baillet, 2020;Ettinger et al, 2014;Gagnepain et al, 2012;Gaston & Marantz, 2018;Gillis et al, 2021;Gwilliams et al, 2020;Gwilliams & Marantz, 2015), and thus appear to be robust to variation in stimulus and experimental task. Cohort entropy, in contrast, produces mixed results.…”

“…Gaston and Marantz (2018) in fact found that their significant cohort entropy effect was no longer significant in a model that controlled for phoneme surprisal, and the other two studies (Ettinger et al, 2014;Kocagoncu et al, 2017) did not conduct such a test. In continuous speech, cohort entropy effects were reported in all studies that tested for them (Brodbeck et al, 2018(Brodbeck et al, , 2021Gillis et al, 2021;Gwilliams et al, 2020), with methods that controlled for effects of phoneme surprisal. We conclude that, in the existing electrophysiology literature, there is strong evidence for phoneme surprisal effects across the board, but for cohort entropy effects only in continuous speech.…”

Envelope reconstruction of speech and music highlights stronger tracking of speech at low frequencies

“…The importance of low-frequency tracking (within the range generally considered "delta" in most studies) for speech is not new [14,29,53], but to our knowledge no study has suggested that low-frequency tracking is stronger for speech than other naturalistic stimuli. Recent studies also found that parietal weighting was increased for tracking phoneme and word surprisal [54] as well as semantic tracking of speech for native speakers but not non-native speakers [55]. The parietal weighting could be indicative of language-specific processing in the posterior temporal lobe [56] which was recently shown to be absent when listening to music [57], but because the activations are broad and without source localization it is difficult to identify the location definitively.…”

Neural tracking as a diagnostic tool to assess the auditory pathway

“…As the prediction accuracies of the forward model are small in magnitude, finding a significant improvement of the linguistic representation over and beyond acoustic representations is statistically challenging (e.g. an improvement of ∼1% corresponds to an increase in prediction accuracy of 3.4×10 −4 using the conservative and restrictive approach as described above Gillis et al (2021b)).…”