Testing the Limits of the Stimulus Reconstruction Approach: Auditory Attention Decoding in a Four-Speaker Free Field Environment

Abstract

Auditory attention can be defined as the cognitive process that enables us to selectively focus on relevant aspects of the acoustic environment while other aspects are ignored. The remarkable ability of the auditory system to focus on one out of several speakers in a multispeaker environment has become known as the cocktail party effect. Although the neural processes underlying selective auditory attention (SAA) are not well understood, it has recently been shown that the cortical representation of a listener’s attended sound stream can be recorded noninvasively from the scalp and that stimulus reconstruction from single trial electroencephalographic (EEG) data enables the decoding of the orientation of auditory attention. The present study extends this approach by evaluating its efficacy in a naturalistic and challenging four-speaker acoustic free field environment, in which the four speakers were spatially separated and presented different but equally salient spoken messages to the listeners. Ten participants were instructed to focus SAA on a spoken prose message in one of the four loudspeakers while ignoring the remaining three streams of prose. Concurrent EEG activity recorded via 128 scalp channels was used for a stimulus reconstruction analysis. The results showed that this approach can be used to decode the orientation of SAA even in a complex and realistic acoustic setting. To confirm that the successful decoding was driven by correspondences between the recorded EEG activity and the attended speech envelopes, the analysis method was validated against randomly constructed sets of surrogate data and by correlations with behavioral data.

Vestigial auriculomotor activity indicates the direction of auditory attention in humans

“…They suggest that patterns of auricular muscle activity might serve as an easily accessible correlate of top-down processing in endogenous modes of attention. As such, the described effects might complement electroencephalographic indices of attentional focus ( de Cheveigné et al, 2018 ; Schäfer et al, 2018 ) in that their sensitivity is exclusively spatial, rather than reflecting a context-specific mixture of modality, feature, location, and object representations. Registration of pinna-orienting might better support near real-time decoding of the attentional focus and, as compared to EEG-based stimulus reconstruction approaches, does not require the exogenous sound source, for example see the discussion in Schäfer et al, 2018 .…”

“…As such, the described effects might complement electroencephalographic indices of attentional focus ( de Cheveigné et al, 2018 ; Schäfer et al, 2018 ) in that their sensitivity is exclusively spatial, rather than reflecting a context-specific mixture of modality, feature, location, and object representations. Registration of pinna-orienting might better support near real-time decoding of the attentional focus and, as compared to EEG-based stimulus reconstruction approaches, does not require the exogenous sound source, for example see the discussion in Schäfer et al, 2018 . Thus, auricular muscle monitoring might support the decoding of auditory attention in technical applications such as attentionally controlled hearing aids that preferentially amplify sounds the user is attempting to listen to.…”

Vestigial Auriculomotor Activity Indicates the Direction of Auditory Attention in Humans

“…“Ear trackers” would complement eye trackers by offering a panoramic rather than narrow frontal range. They would complement electroencephalographic measures (de Cheveigné et al, 2018; Schäfer, Corona-Strauss, Hannemann, Hillyard, & Strauss, 2018) in that their sensitivity is exclusively spatial, rather than reflecting a context-specific mixture of modality, feature, location, and object representations. Moreover, EMG promises a decoding in near real–time (range of hundreds of milliseconds) and would not require information regarding the exogenous sound source.…”

EEG-Based Auditory Attention Detection and Its Possible Future Applications for Passive BCI

“…The vast majority of the studies that explored EEG-based AAD performances used two concurrent spatially separated talkers but some of them have explored the impact of speaker number and their location in auditory scene (Schäfer et al, 2018), background noise (Das et al, 2018), reverberation (Fuglsang et al, 2017), number of EEG electrodes (Mirkovic et al, 2015;Bleichner et al, 2016), or even their location (Fiedler et al, 2017) on the performance of AAD algorithms.…”