An EEG-Based Brain Computer Interface for Emotion Recognition and Its Application in Patients with Disorder of Consciousness

Abstract

Recognizing human emotions based on electroencephalogram (EEG) signals has received a great deal of attentions. Most of the existing studies focused on offline analysis, and real-time emotion recognition using a brain computer interface (BCI) approach remains to be further investigated. In this paper, we proposed an EEG-based BCI system for emotion recognition. Specifically, two classes of video clips that represented positive and negative emotions were presented to the subjects one by one, while the EEG data were collected and processed simultaneously, and instant feedback was provided after each clip. Ten healthy subjects participated in the experiment and achieved a high average online accuracy of 91.5% ± 6.34%. The experimental results demonstrated that the subjects emotions had been sufficiently evoked and efficiently recognized by our system. Clinically, patients with disorder of consciousness (DOC), such as coma, vegetative state, minimally conscious state and emergence minimally conscious state, suffer from motor impairment and generally cannot provide adequate emotion expressions. Consequently, doctors have difficulty in detecting the emotional states of these patients. Therefore, we applied our emotion recognition BCI system to patients with DOC. Eight DOC patients participated in our experiment, and three of them achieved significant online accuracy. The experimental results show that the proposed BCI system could be a promising tool to detect the emotional states of patients with DOC.

Advances in Multimodal Emotion Recognition Based on Brain–Computer Interfaces

“…Multisensory channel integration to induce subjects’ emotions, especially video material that involves emotion recognition evoked by visual and auditory stimuli, will be a future trend. For audio-visual emotion recognition, a real-time BCI system to identify the emotions of patients with consciousness disorders was proposed [ 10 ]. Specifically, two classes of video clips were used to induce positive and negative emotions sequentially in the subjects, and the EEG data were collected and processed simultaneously.…”

“…The existing research on emotion recognition mainly involves offline analysis. For the first time, Huang et al [ 10 ] applied an aBCI online system to the emotion recognition of patients with disorders of consciousness. Using this system, they were able to successfully induce and detect the emotional characteristics of some patients with consciousness disorders in real time.…”

“…Li et al [ 9 ] proposed that in addition to combining different input signals, emotion recognition should include a variety of heterogeneous sensory stimuli (such as audio-visual stimulation) to induce emotions. Many studies [ 10 , 11 ] have shown that integrating heterogeneous sensory stimuli can enhance brain patterns and further improve brain–computer interface performance.…”

Assessing the Effectiveness of Automated Emotion Recognition in Adults and Children for Clinical Investigation

“…In addition, studies using physiological signals to recognize emotional states such as electroencephalogram (EEG)-based brain-computer interface systems (BCI) are also providing interesting results, and there is promise for use in a number of real-world applications. Huang et al (2019) showed participants video clips with negative and positive valence while recording EEG. The EEG-based BCI system successfully induced and recognized positive and negative emotional states in patients with Disorders of Consciousness.…”

Improved Deep Feature Learning by Synchronization Measurements for Multi-Channel EEG Emotion Recognition

“…Pan et al [25] used the emotion-related common spatial pattern and differential entropy features of five frequency bands in their research. To acquire the time-varying characteristics reflected by EEG frequency data, the short-time Fourier transform was used to analyze EEG signals [26,27]. As EEG signals are time-varying, researchers proposed new methods to obtain additional information by combining time and frequency-domain features.…”