Author response: EEG-based detection of the locus of auditory attention with convolutional neural networks

Vandecappelle, Servaas; Deckers, Lucas; Das, Neetha; Ansari, Amir H.; Bertrand, Alexander; Francart, Tom

doi:10.7554/elife.56481.sa2

Cited by 5 publications

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarly in AAD paradigm, convolutional neural network (CNN) based models were proposed where the stimulus reconstruction algorithm was implemented using the CNN model to infer attention (Ciccarelli et al, 2019;de Taillez et al, 2020). A direct classification of attention which bypasses the regression task of stimulus reconstruction, instead classifies whether the attention is to speaker 1 or speaker 2 directly was proposed in Ciccarelli et al (2019) and Vandecappelle et al (2021). In a non-competing speaker experiment, classifying attention as successful vs unsuccessful or match vs mismatch was further addressed in Monesi et al (2020) and Tian and Ma (2020).…”

Section: Introductionmentioning

confidence: 99%

Extracting the Auditory Attention in a Dual-Speaker Scenario From EEG Using a Joint CNN-LSTM Model

et al. 2021

View full text Add to dashboard Cite

Human brain performs remarkably well in segregating a particular speaker from interfering ones in a multispeaker scenario. We can quantitatively evaluate the segregation capability by modeling a relationship between the speech signals present in an auditory scene, and the listener's cortical signals measured using electroencephalography (EEG). This has opened up avenues to integrate neuro-feedback into hearing aids where the device can infer user's attention and enhance the attended speaker. Commonly used algorithms to infer the auditory attention are based on linear systems theory where cues such as speech envelopes are mapped on to the EEG signals. Here, we present a joint convolutional neural network (CNN)—long short-term memory (LSTM) model to infer the auditory attention. Our joint CNN-LSTM model takes the EEG signals and the spectrogram of the multiple speakers as inputs and classifies the attention to one of the speakers. We evaluated the reliability of our network using three different datasets comprising of 61 subjects, where each subject undertook a dual-speaker experiment. The three datasets analyzed corresponded to speech stimuli presented in three different languages namely German, Danish, and Dutch. Using the proposed joint CNN-LSTM model, we obtained a median decoding accuracy of 77.2% at a trial duration of 3 s. Furthermore, we evaluated the amount of sparsity that the model can tolerate by means of magnitude pruning and found a tolerance of up to 50% sparsity without substantial loss of decoding accuracy.

show abstract

Section: Introductionmentioning

confidence: 99%

Extracting the Auditory Attention in a Dual-Speaker Scenario From EEG Using a Joint CNN-LSTM Model

et al. 2021

View full text Add to dashboard Cite

show abstract

Fast EEG-based decoding of the directional focus of auditory attention using common spatial patterns

Geirnaert

Francart

Bertrand

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Objective: Noise reduction algorithms in current hearing prostheses lack information about the sound source a user attends to when multiple sources are present. To resolve this issue, they can be complemented with auditory attention decoding (AAD) algorithms, which decode the attention directly from the brain using electroencephalography (EEG) sensors. State-of-theart AAD algorithms employ a stimulus reconstruction approach, in which the envelope of the attended source is reconstructed from the EEG and correlated with the envelopes of the individual sources. This approach, however, performs poorly on short signal segments, while longer segments yield impractically long detection delays when the user switches attention. Methods: We propose decoding the directional focus of attention using filterbank common spatial pattern filters (FB-CSP) as an alternative AAD paradigm, which does not require access to the clean source envelopes. Results: The proposed FB-CSP approach outperforms both the traditional stimulus reconstruction approach, as well as a convolutional neural network approach on the same task. We achieve a high accuracy (80% for 1 s windows and 70% for quasiinstantaneous decisions), which is sufficient to reach minimal expected switch durations below 4 s. We also demonstrate that the method can be used on unlabeled data from an unseen subject and with only a subset of EEG channels located around the ear to emulate a wearable EEG setup. Conclusion: The proposed FB-CSP method provides fast and accurate decoding of the directional focus of auditory attention. Significance: The high accuracy on very short data segments is a major step forward towards practical neuro-steered hearing prostheses.Index Terms-auditory attention decoding, directional focus of attention, brain-computer interface, common spatial pattern filter, electroencephalography, neuro-steered hearing prosthesis

show abstract

Envelope reconstruction of speech and music highlights unique tracking of speech at low frequencies

Zuk

Murphy

Reilly

et al. 2021

Preprint

View full text Add to dashboard Cite

The human brain tracks amplitude fluctuations of both speech and music, which reflects acoustic processing in addition to the processing of higher-order features and one’s cognitive state. Comparing neural tracking of speech and music envelopes can elucidate stimulus-general mechanisms, but direct comparisons are confounded by differences in their envelope spectra. Here, we use a novel method of frequency-constrained reconstruction of stimulus envelopes using EEG recorded during passive listening. We expected to see music reconstruction match speech in a narrow range of frequencies, but instead we found that speech was reconstructed better than music for all frequencies we examined. Additionally, speech envelope tracking at low frequencies, below 1 Hz, was uniquely associated with increased weighting over parietal channels. Our results highlight the importance of low-frequency speech tracking and its origin from speech-specific processing in the brain.

show abstract

Author response: EEG-based detection of the locus of auditory attention with convolutional neural networks

Cited by 5 publications

References 0 publications

Extracting the Auditory Attention in a Dual-Speaker Scenario From EEG Using a Joint CNN-LSTM Model

Extracting the Auditory Attention in a Dual-Speaker Scenario From EEG Using a Joint CNN-LSTM Model

Fast EEG-based decoding of the directional focus of auditory attention using common spatial patterns

Envelope reconstruction of speech and music highlights unique tracking of speech at low frequencies

Contact Info

Product

Resources

About