The Study of Influence of Sound on Visual ERP-Based Brain Computer Interface

Abstract

The performance of the event-related potential (ERP)-based brain–computer interface (BCI) declines when applying it into the real environment, which limits the generality of the BCI. The sound is a common noise in daily life, and whether it has influence on this decline is unknown. This study designs a visual-auditory BCI task that requires the subject to focus on the visual interface to output commands and simultaneously count number according to an auditory story. The story is played at three speeds to cause different workloads. Data collected under the same or different workloads are used to train and test classifiers. The results show that when the speed of playing the story increases, the amplitudes of P300 and N200 potentials decrease by 0.86 μV (p = 0.0239) and 0.69 μV (p = 0.0158) in occipital-parietal area, leading to a 5.95% decline (p = 0.0101) of accuracy and 9.53 bits/min decline (p = 0.0416) of information transfer rate. The classifier that is trained by the high workload data achieves higher accuracy than the one trained by the low workload if using the high workload data to test the performance. The result indicates that the sound could affect the visual ERP-BCI by increasing the workload. The large similarity of the training data and testing data is as important as the amplitudes of the ERP on obtaining high performance, which gives us an insight on how make to the ERP-BCI generalized.

Effect of Distracting Background Speech in an Auditory Brain–Computer Interface

“…The novelty of this study was twofold: (i) there is no previous study that has evaluated the effect of a background speech in an auditory ERP-BCI and (ii) it was based on the use of two control conditions (C1 and C2): one to study the simple presence of the background audio (C1 vs. C2 and C3), and a second to study the effect of the secondary task of attention (C2 vs. C3). This complete control design was not used in previous studies involving distractors in an ERP-BCI (i.e., Käthner et al [ 15 ] or Xu et al [ 16 ], who used a visual ERP-BCI, and Zhou et al [ 17 ], who used an auditory ERP-BCI). First, Käthner et al [ 15 ] lacked control conditions for the presence of audio (i.e., the condition C1).…”

“…First, Käthner et al [ 15 ] lacked control conditions for the presence of audio (i.e., the condition C1). Secondly, Xu et al [ 16 ] also did not evaluate the presence of audio or a secondary attentional task as it did not have a condition without audio or with audio, but no attention (i.e., C1 or C2, respectively). Finally, Zhou et al [ 17 ] did not present a condition with attention (i.e., C3).…”

“…Indeed, distractors, such as external stimuli not implicated in the control of the interface, or conflicting attentional demands could increase the workload on the user and directly affect the performance of the BCI. The effects of distractors on BCI controls have been studied previously using three types of electroencephalographic (EEG) signal: steady-state visual evoked potentials (SSVEP) [ 10 ], event-related (de)synchronization phenomena (ERD/ERS) [ 11 , 12 , 13 ], and event-related potentials (ERP) [ 14 , 15 , 16 , 17 ]. These studies have shown that distractors can affect the performance, the ERP waveform, or the subjective experience of the user.…”

“…To the best of our knowledge, only three previous studies based on ERP-BCI have examined the effect of background: two were based on visual ERP-BCI and studied the effect of a background talk [ 15 , 16 ], and only one was based on auditory ERP-BCI, examining the effect of background music [ 17 ]. First, Käthner et al [ 15 ] found a lower performance when participants had to pay attention to two concurrent stories presented over headphones (auditory attentional condition) compared with when they ignored them (auditory non-attentional condition).…”

“…The grand average of waveforms for the attentional condition was significantly smaller than that of the non-attentional condition. Later, Xu et al [ 16 ] showed that requiring the user to listen to an audio story at different speeds decreased performance as the user workload increased (the faster the audio story, the greater the workload). They also found that the amplitude of the P300 component, in the parietal and occipital regions, was negatively related to the increase in difficulty of additional auditory tasks.…”