EEG-based detection of the locus of auditory attention with convolutional neural networks

Vandecappelle, Servaas; Deckers, Lucas; Das, Neetha; Ansari, Amir H.; Bertrand, Alexander; Francart, Tom

doi:10.7554/elife.56481

Cited by 57 publications

(86 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is noted that non-linear models show much better performance than linear models, particularly in low-latency settings (de Taillez et al, 2017 ; Deckers et al, 2018 ; Ciccarelli et al, 2019 ; Vandecappelle et al, 2021 ). Since the other reported non-linear models are reported on different datasets, a direct comparison with CMAA is not straightforward.…”

Section: Discussionmentioning

confidence: 99%

“…AAD is usually formulated as a binary classification problem in a two-speaker scenario (de Taillez et al, 2017 ; Deckers et al, 2018 ; Vandecappelle et al, 2021 ). First, the CSP method was used for discriminative feature extraction of the original EEG signals.…”

Section: Methodsmentioning

confidence: 99%

“…All the EEG data were bandpass-filtered between 2 and 32 Hz with a finite impulse response (FIR) filter and subsequently downsampled to 70 Hz. The frequency range was chosen based on the previous non-linear AAD studies (de Taillez et al, 2017 ; Deckers et al, 2018 ; Vandecappelle et al, 2021 ). Finally, the EEG data channels were normalized to ensure zero mean and unit variance for each trial.…”

Section: Methodsmentioning

confidence: 99%

“…Each subband was further processed with a power-law compression with an exponent of 0.6. The subband envelopes were then added to generate a broadband envelope, which was filtered with the same filter as used for the EEG recordings and then downsampled to 70 Hz to match the EEG data (Deckers et al, 2018 ; Vandecappelle et al, 2021 ), denoted as H-LP in Figure 1 . Finally, the stimulus amplitudes in each speech stream within each trial were normalized to have the same RMS intensity.…”

Section: Methodsmentioning

confidence: 99%

“…de Taillez et al ( 2017 ) studied a non-linear neural network for mapping EEG signals to speech envelopes in a cocktail party scenario and showed that it outperforms the linear model baseline. Following a similar approach, convolutional neural network (CNN) models (Deckers et al, 2018 ; Ciccarelli et al, 2019 ; Cai et al, 2020 ; Vandecappelle et al, 2021 ) were studied to detect the attended speakers. However, these non-linear AAD approaches neglect valuable temporal information of EEG signals and more advanced decoding strategies are needed to realize robust real-time AAD.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Auditory Attention Detection via Cross-Modal Attention

Cai

et al. 2021

Front. Neurosci.

View full text Add to dashboard Cite

Humans show a remarkable perceptual ability to select the speech stream of interest among multiple competing speakers. Previous studies demonstrated that auditory attention detection (AAD) can infer which speaker is attended by analyzing a listener's electroencephalography (EEG) activities. However, previous AAD approaches perform poorly on short signal segments, more advanced decoding strategies are needed to realize robust real-time AAD. In this study, we propose a novel approach, i.e., cross-modal attention-based AAD (CMAA), to exploit the discriminative features and the correlation between audio and EEG signals. With this mechanism, we hope to dynamically adapt the interactions and fuse cross-modal information by directly attending to audio and EEG features, thereby detecting the auditory attention activities manifested in brain signals. We also validate the CMAA model through data visualization and comprehensive experiments on a publicly available database. Experiments show that the CMAA achieves accuracy values of 82.8, 86.4, and 87.6% for 1-, 2-, and 5-s decision windows under anechoic conditions, respectively; for a 2-s decision window, it achieves an average of 84.1% under real-world reverberant conditions. The proposed CMAA network not only achieves better performance than the conventional linear model, but also outperforms the state-of-the-art non-linear approaches. These results and data visualization suggest that the CMAA model can dynamically adapt the interactions and fuse cross-modal information by directly attending to audio and EEG features in order to improve the AAD performance.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%