Modeling the relationship between acoustic stimulus and EEG with a dilated convolutional neural network

Accou, Bernd; Monesi, Mohammad Jalilpour; Montoya, J. Andres; hamme, Hugo Van; Francart, Tom

doi:10.23919/eusipco47968.2020.9287417

Cited by 29 publications

(79 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The EEG data analysed in this paper is part of a large existing dataset (Accou et al., 2021; Monesi et al., 2020) that contains EEG responses to continuous Flemish speech for young normal hearing participants (not publicly available). A subset of 46 subjects (41 females, 5 males) was selected based on presentation with the same experimental conditions.…”

Section: Methodsmentioning

confidence: 99%

Neural tracking of the fundamental frequency of the voice: The effect of voice characteristics

Canneyt

Wouters

Francart

2021

Eur J of Neuroscience

Self Cite

View full text Add to dashboard Cite

Traditional electrophysiological methods to study temporal auditory processing of the fundamental frequency of the voice (f0) often use unnaturally repetitive stimuli. In this study, we investigated f0 processing of meaningful continuous speech. EEG responses evoked by stories in quiet were analysed with a novel method based on linear modelling that characterizes the neural tracking of the f0. We studied both the strength and the spatio‐temporal properties of the f0‐tracking response. Moreover, different samples of continuous speech (six stories by four speakers: two male and two female) were used to investigate the effect of voice characteristics on the f0 response. The results indicated that response strength is inversely related to f0 frequency and rate of f0 change throughout the story. As a result, the male‐narrated stories in this study (low and steady f0) evoked stronger f0‐tracking compared to female‐narrated stories (high and variable f0), for which many responses were not significant. The spatio‐temporal analysis revealed that f0‐tracking response generators were not fixed in the brainstem but were voice‐dependent as well. Voices with high and variable f0 evoked subcortically dominated responses with a latency between 7 and 12 ms. Voices with low and steady f0 evoked responses that are both subcortically (latency of 13–15 ms) and cortically (latency of 23–26 ms) generated, with the right primary auditory cortex as a likely cortical source. Finally, additional experiments revealed that response strength greatly improves for voices with strong higher harmonics, which is particularly useful to boost the small responses evoked by voices with high f0.

show abstract

Section: Methodsmentioning

confidence: 99%

Neural tracking of the fundamental frequency of the voice: The effect of voice characteristics

Canneyt

Wouters

Francart

2021

Eur J of Neuroscience

Self Cite

View full text Add to dashboard Cite

show abstract

“…The electroencephalography (EEG) data of 29 young normal-hearing individuals (22 ♀) were analysed. The data were originally collected for other studies (Accou et al, 2020;Monesi et al, 2020). Participant age varied between 18 and 25 years old (mean±std= 20.81 ± 1.94 years).…”

Section: Participant Detailsmentioning

confidence: 99%

Neural markers of speech comprehension: measuring EEG tracking of linguistic speech representations, controlling the speech acoustics

Gillis

Vanthornhout

Simon

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

When listening to speech, brain responses time-lock to acoustic events in the stimulus. Recent studies have also reported that cortical responses track linguistic representations of speech. However, tracking of these representations is often described without controlling for acoustic properties. Therefore, the response to these linguistic representations might reflect unaccounted acoustic processing rather than language processing. Here we tested several recently proposed linguistic representations, using audiobook speech, while controlling for acoustic and other linguistic representations. Indeed, some of these linguistic representations were not significantly tracked after controlling for acoustic properties. However, phoneme surprisal, cohort entropy, word surprisal and word frequency were significantly tracked over and beyond acoustic properties. Additionally, these linguistic representations are tracked similarly across different stories, spoken by different readers. Together, this suggests that these representations characterize processing of the linguistic content of speech and might allow a behaviour-free evaluation of the speech intelligibility.

show abstract

“…Section 1 of figure 3 shows the results of a typical forward modelling analysis for f0-tracking, obtained using the methods described in Van Canneyt et al (2021c). The data set used for this visualisation (and all others in figure 3) contained 64-channel EEG data from 32 young normal-hearing subjects measured in response to male-narrated speech (dataset from Accou et al, 2021) (2020) found no age effects using MEG, which is most sensitive to cortical sources. In contrast, Van Canneyt et al…”

Section: Neural Tracking Of the F0mentioning

confidence: 99%

“…The copyright holder for this this version posted November 27, 2021. ; https://doi.org/10. 1101/2021.11.26.470129 doi: bioRxiv preprint Zan et al, 2020;Kaufeld et al, 2020) and neural networks (Katthi et al, 2020;Accou et al, 2021), but these will not be discussed further.…”

Section: Introductionmentioning

confidence: 99%

Neural tracking as a diagnostic tool to assess the auditory pathway

Canneyt

Gillis

Vanthornhout

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The neural tracking framework enables the analysis of neural responses (EEG) to continuous natural speech, e.g., a story or a podcast. This allows for objective investigation of a range of auditory and linguistic processes in the brain during natural speech perception. This approach is more ecologically valid than traditional auditory evoked responses and has great potential for both research and clinical applications. In this article, we review the neural tracking framework and highlight three prominent examples of neural tracking analyses. This includes the neural tracking of the fundamental frequency of the voice (f0), the speech envelope and linguistic features. Each of these analyses provides a unique point of view into the hierarchical stages of speech processing in the human brain. f0-tracking assesses the encoding of fine temporal information in the early stages of the auditory pathway, i.e. from the auditory periphery up to early processing in the primary auditory cortex. This fundamental processing in (mostly) subcortical stages forms the foundation of speech perception in the cortex. Envelope tracking reflects bottom-up and top-down speech-related processes in the auditory cortex, and is likely necessary but not sufficient for speech intelligibility. To study neural processes more directly related to speech intelligibility, neural tracking of linguistic features can be used. This analysis focuses on the encoding of linguistic features (e.g. word or phoneme surprisal) in the brain. Together these analyses form a multi-faceted and time-effective objective assessment of the auditory and linguistic processing of an individual.

show abstract

Modeling the relationship between acoustic stimulus and EEG with a dilated convolutional neural network

Cited by 29 publications

References 13 publications

Neural tracking of the fundamental frequency of the voice: The effect of voice characteristics

Neural tracking of the fundamental frequency of the voice: The effect of voice characteristics

Neural markers of speech comprehension: measuring EEG tracking of linguistic speech representations, controlling the speech acoustics

Neural tracking as a diagnostic tool to assess the auditory pathway

Contact Info

Product

Resources

About