Neural tracking to go: auditory attention decoding and saliency detection with mobile EEG

Abstract

Objective. Neuro-steered assistive technologies have been suggested to offer a major advancement in future devices like neuro-steered hearing aids. Auditory attention decoding methods would in that case allow for identification of an attended speaker within complex auditory environments, exclusively from neural data. Decoding the attended speaker using neural information has so far only been done in controlled laboratory settings. Yet, it is known that ever-present factors like distraction and movement are reflected in the neural signal parameters related to attention. Approach. Thus, in the current study we applied a two-competing speaker paradigm to investigate performance of a commonly applied EEG-based auditory attention decoding (AAD) model outside of the laboratory during leisure walking and distraction. Unique environmental sounds were added to the auditory scene and served as distractor events. Main results. The current study shows, for the first time, that the attended speaker can be accurately decoded during natural movement. At a temporal resolution of as short as 5-seconds and without artifact attenuation, decoding was found to be significantly above chance level. Further, as hypothesized, we found a decrease in attention to the to-be-attended and the to-be-ignored speech stream after the occurrence of a salient event. Additionally, we demonstrate that it is possible to predict neural correlates of distraction with a computational model of auditory saliency based on acoustic features. Conclusion. Taken together, our study shows that auditory attention tracking outside of the laboratory in ecologically valid conditions is feasible and a step towards the development of future neural-steered hearing aids.

Ear-EEG Measures of Auditory Attention to Continuous Speech

“…Consequently, the decoding accuracy may benefit more from artifact correction when shorter segments of data are used. In Jaeger et al (2020) , ASR improved classification of shorter data segments, whereas Straetmans et al (2022) showed that even for data segments as short as 5 s, decoding accuracies were not increased when the data was cleaned with ASR. We speculate that these heterogenous results could reflect the quality of the calibration data that were used to perform ASR.…”

“…In Jaeger et al (2020) , the calibration data were extracted while participants performed a task (i.e., the competing speaker paradigm). In Straetmans et al (2022) , the calibration data were acquired while participants were seated without performing any task. It is known that good calibration data are crucial when performing ASR ( Blum et al, 2019 ).…”

Brains in Sync: Practical Guideline for Parent–Infant EEG During Natural Interaction

“…There are - as far as we are aware – no dual-EEG studies with infants that use mobile EEG systems, but it would be a major opportunity for future purposes. Portable systems have no cables hanging around that could distract the infant, it can be used outside the laboratory, it allows more freedom of movements ( Lau-Zhu et al, 2019 ) and it is possible to track head motion at the same time (see for example, Straetmans et al, 2022 ). However, mobile EEG systems have their own challenges; they often have fewer electrodes, dry-type electrodes (without gel), and can suffer from connectivity issues (not saving data).…”

Eye-blink patterns reflect attention to continuous speech

“…As Huet et al (2021) pointed out, this approach cannot capture attentional fluctuations over time, even though they have been described in detail (e.g., Jaeger et al, 2020). The difficulty of tracking the dynamics of attentional processes is also reflected in observations that salient events embedded in the ignored speech stream can capture one's attention and shift the attention to a previously ignored speaker (Holtze et al, 2021;Huang & Elhilali, 2020;Straetmans et al, 2022). Consequently, an implicit behavioral measure of attention is needed.…”