EEG-based auditory attention detection: boundary conditions for background noise and speaker positions

Abstract

Objective. A listener's neural responses can be decoded to identify the speaker the person is attending to in a cocktail party environment. Such auditory attention detection methods have the potential to provide noise suppression algorithms in hearing devices with information about the listener's attention. A challenge is the effect of noise and other acoustic conditions that can reduce the attention detection accuracy. Specifically, noise can impact the ability of the person to segregate the sound sources and perform selective attention, as well as the external signal processing necessary to decode the attention effectively. The aim of this work is to systematically analyze the effect of noise level and speaker position on attention decoding accuracy. Approach. 28 subjects participated in the experiment. Auditory stimuli consisted of stories narrated by different speakers from 2 different locations, along with surrounding multitalker background babble. EEG signals of the subjects were recorded while they focused on one story and ignored the other. The strength of the babble noise as well as the spatial separation between the two speakers were varied between presentations. Spatio-temporal decoders were trained for each subject, and applied to decode attention of the subjects from every 30s segment of data. Behavioral speech recognition thresholds were obtained for the different speaker separations. Main results. Both the background noise level and the angular separation between speakers affected attention decoding accuracy. Remarkably, attention decoding performance was seen to increase with the inclusion of moderate background noise (versus no noise), while across the different noise conditions performance dropped significantly with increasing noise level. We also observed that decoding accuracy improved with increasing speaker separation, exhibiting the advantage of spatial release from masking. Furthermore, the effect of speaker separation on the decoding accuracy became stronger when the background noise level increased. A significant correlation between speech intelligibility and attention decoding accuracy was found across conditions. Significance. This work shows how the background noise level and relative positions of competing talkers impact attention decoding accuracy. It indicates in which circumstances a neuro-steered noise suppression system may need to operate, in function of acoustic conditions. It also indicates the boundary conditions for the operation of EEG-based attention detection systems in neuro-steered hearing prostheses.

Top-down modulation of neural envelope tracking: the interplay with behavioral, self-report and neural measures of listening effort

“…However, young NH participants show differences in neural envelope tracking that can not be fully attributed to changes in SNR or speech understanding. In addition, recent studies have observed a lower envelope tracking for equally intelligible sentences presented without a masker compared to sentences presented at a high SNR (Das et al, 2018;Lesenfants et al, 2019). We hypothesize that these two unexplained differences in envelope tracking are related to changes in listening effort based on the findings of following behavioral studies.…”

“…Based on previous studies, we expected to find lower envelope tracking for speech without a masker compared to speech presented at a high SNR due to lower effort. For example, Das et al (2018) found lower neural envelope tracking for stories presented without a masker compared to stories presented at -1.1 dB SNR. Similarly, Lesenfants et al (2019) observed lower envelope tracking in the theta-band for matrix sentences presented without a masker compared to sentences presented at -3.5, -1, -0.5 or 2.5 dB SNR.…”

An Interpretable Performance Metric for Auditory Attention Decoding Algorithms in a Context of Neuro-Steered Gain Control

“…1) Description of the two variants: Given a training set of M data windows, in the first variant of [6] (also adopted in, e.g., [8]), per-window (corresponding to decision window length τ ) decoders are computed, after which the M decoders are averaged to obtain one final decoder. The second variant of [10] (also adopted in, e.g., [12], [17], [18]), first averages the M per-window autocorrelation matrices (or equivalently: the windows are all concatenated) to train a single decoder across all training windows simultaneously. Similarly to [10], L 2 -norm regularization is added to the former method, in order to avoid overfitting effects due to the small amount of data per decoder.…”

“…Later, also alternative pipelines were presented based on other source separation algorithms [13]- [16]. Furthermore, the effects of different boundary conditions, such as speaker positions or noisy and reverberant conditions, have already extensively studied as well (e.g., [17]- [19]).…”

The interplay of top-down focal attention and the cortical tracking of speech

“…The correlation between the reconstructed/predicted artificial time-course and the actual values is then a measure of cortical speech tracking. Measures of cortical speech tracking quantify speech processing within the brain opening doors to, for example, an objective measure of an individual speech understanding [4][5][6][7] or auditory attention decoding in a cocktail party scenario [8][9][10] .…”