Data Augmentation for EEG-Based Emotion Recognition with Deep Convolutional Neural Networks

Wang, Fang; Zhong, Sheng-hua; Peng, Jue; Jiang, Jianmin; Liu, Yanping

doi:10.1007/978-3-319-73600-6_8

Cited by 178 publications

(119 citation statements)

References 26 publications

Supporting

Mentioning

109

Contrasting

Order By: Relevance

“…Also, all three are from 2018, perhaps showing an emerging interest in data augmentation. First, in [192], Gaussian noise was added to the training data to obtain new examples. This approach was tested on two different public datasets for emotion classification (SEED [227] and MAHNOB-HCI [159] In [152], the authors explicitly targeted the class imbalance problem of under-represented sleep stages by generating Fourier transform (FT) surrogates of raw EEG data on the CAPSLPDB dataset.…”

Section: Data Augmentationmentioning

confidence: 99%

Deep learning-based electroencephalography analysis: a systematic review

et al. 2019

View full text Add to dashboard Cite

Context. Electroencephalography (EEG) is a complex signal and can require several years of training, as well as advanced signal processing and feature extraction methodologies to be correctly interpreted. Recently, deep learning (DL) has shown great promise in helping make sense of EEG signals due to its capacity to learn good feature representations from raw data. Whether DL truly presents advantages as compared to more traditional EEG processing approaches, however, remains an open question.Objective. In this work, we review 156 papers that apply DL to EEG, published between January 2010 and July 2018, and spanning different application domains such as epilepsy, sleep, braincomputer interfacing, and cognitive and affective monitoring. We extract trends and highlight interesting approaches from this large body of literature in order to inform future research and formulate recommendations.Methods. Major databases spanning the fields of science and engineering were queried to identify relevant studies published in scientific journals, conferences, and electronic preprint repositories. Various data items were extracted for each study pertaining to 1) the data, 2) the preprocessing methodology, 3) the DL design choices, 4) the results, and 5) the reproducibility of the experiments. These items were then analyzed one by one to uncover trends. * The first two authors contributed equally to this work.Significance. To help the community progress and share work more effectively, we provide a list of recommendations for future studies. We also make our summary table of DL and EEG papers available and invite authors of published work to contribute to it directly.

show abstract

Section: Data Augmentationmentioning

confidence: 99%

Deep learning-based electroencephalography analysis: a systematic review

et al. 2019

View full text Add to dashboard Cite

show abstract

“…This also prevents the network from overfitting due to data repetition. This technique was similarly applied in [70].…”

Section: Data Reshapingmentioning

confidence: 99%

Investigating the Use of Pretrained Convolutional Neural Network on Cross-Subject and Cross-Dataset EEG Emotion Recognition

Çimtay

Ekmekcioǧlu

2020

Sensors

183

108

View full text Add to dashboard Cite

The electroencephalogram (EEG) has great attraction in emotion recognition studies due to its resistance to deceptive actions of humans. This is one of the most significant advantages of brain signals in comparison to visual or speech signals in the emotion recognition context. A major challenge in EEG-based emotion recognition is that EEG recordings exhibit varying distributions for different people as well as for the same person at different time instances. This nonstationary nature of EEG limits the accuracy of it when subject independency is the priority. The aim of this study is to increase the subject-independent recognition accuracy by exploiting pretrained state-of-the-art Convolutional Neural Network (CNN) architectures. Unlike similar studies that extract spectral band power features from the EEG readings, raw EEG data is used in our study after applying windowing, pre-adjustments and normalization. Removing manual feature extraction from the training system overcomes the risk of eliminating hidden features in the raw data and helps leverage the deep neural network’s power in uncovering unknown features. To improve the classification accuracy further, a median filter is used to eliminate the false detections along a prediction interval of emotions. This method yields a mean cross-subject accuracy of 86.56% and 78.34% on the Shanghai Jiao Tong University Emotion EEG Dataset (SEED) for two and three emotion classes, respectively. It also yields a mean cross-subject accuracy of 72.81% on the Database for Emotion Analysis using Physiological Signals (DEAP) and 81.8% on the Loughborough University Multimodal Emotion Dataset (LUMED) for two emotion classes. Furthermore, the recognition model that has been trained using the SEED dataset was tested with the DEAP dataset, which yields a mean prediction accuracy of 58.1% across all subjects and emotion classes. Results show that in terms of classification accuracy, the proposed approach is superior to, or on par with, the reference subject-independent EEG emotion recognition studies identified in literature and has limited complexity due to the elimination of the need for feature extraction.

show abstract

“…This results in sub-segments that are not completely independent from each other, so that we ensured that each 6 sub-segments of one trial fall into the same fold of cross validation. Other publications that work with deep learning use even less independent scenarios of data augmentation with EEG data ( 91 ). In such a scenario one would create 999 segments out of the 6 s by using a sliding window that shifts over the segment and creates 1 s segments starting from each sample point.…”

Section: Discussionmentioning

confidence: 99%

HD-EEG Based Classification of Motor-Imagery Related Activity in Patients With Spinal Cord Injury

et al. 2018

View full text Add to dashboard Cite

Brain computer interfaces (BCIs) are thought to revolutionize rehabilitation after SCI, e.g., by controlling neuroprostheses, exoskeletons, functional electrical stimulation, or a combination of these components. However, most BCI research was performed in healthy volunteers and it is unknown whether these results can be translated to patients with spinal cord injury because of neuroplasticity. We sought to examine whether high-density EEG (HD-EEG) could improve the performance of motor-imagery classification in patients with SCI. We recorded HD-EEG with 256 channels in 22 healthy controls and 7 patients with 14 recordings (4 patients had more than one recording) in an event related design. Participants were instructed acoustically to either imagine, execute, or observe foot and hand movements, or to rest. We calculated Fast Fourier Transform (FFT) and full frequency directed transfer function (ffDTF) for each condition and classified conditions pairwise with support vector machines when using only 2 channels over the sensorimotor area, full 10-20 montage, high-density montage of the sensorimotor cortex, and full HD-montage. Classification accuracies were comparable between patients and controls, with an advantage for controls for classifications that involved the foot movement condition. Full montages led to better results for both groups (p < 0.001), and classification accuracies were higher for FFT than for ffDTF (p < 0.001), for which the feature vector might be too long. However, full-montage 10–20 montage was comparable to high-density configurations. Motor-imagery driven control of neuroprostheses or BCI systems may perform as well in patients as in healthy volunteers with adequate technical configuration. We suggest the use of a whole-head montage and analysis of a broad frequency range.

show abstract

Data Augmentation for EEG-Based Emotion Recognition with Deep Convolutional Neural Networks

Cited by 178 publications

References 26 publications

Deep learning-based electroencephalography analysis: a systematic review

Deep learning-based electroencephalography analysis: a systematic review

Investigating the Use of Pretrained Convolutional Neural Network on Cross-Subject and Cross-Dataset EEG Emotion Recognition

HD-EEG Based Classification of Motor-Imagery Related Activity in Patients With Spinal Cord Injury

Contact Info

Product

Resources

About