Discrimination of Genuine and Acted Emotional Expressions Using EEG Signal and Machine Learning

Abstract

We present here one of the first studies that attempt to differentiate between genuine and acted emotional expressions, using EEG data. We present the first EEG dataset (available here) with recordings of subjects with genuine and fake emotional expressions. We build our experimental paradigm for classification of smiles; genuine smiles, fake/acted smiles and neutral expression. We propose multiple methods to extract intrinsic features from three EEG emotional expressions; genuine, neutral, and fake/acted smile. We extracted EEG features using three time-frequency analysis methods: discrete wavelet transforms (DWT), empirical mode decomposition (EMD), and incorporating DWT into EMD (DWT-EMD) at three frequency bands. We then evaluated the proposed methods using several classifiers including, k-nearest neighbors (KNN), support vector machine (SVM), and artificial neural network (ANN). We carried out an experimental paradigm on 28-subjects underwent three types of emotional expressions, genuine, neutral and fake/acted. The results showed that incorporating DWT into EMD extracted more hidden features than sole DWT or sole EMD method. The power spectral feature extracted by DWT, EMD, and DWT-EMD showed different neural patterns across the three emotional expressions at all the frequency bands. We performed binary classification experiments and achieved acceptable accuracy reaching a maximum of 84% in all type of emotions, classifiers and bands using sole DWT or EMD. Meanwhile, a combination of DWT-EMD achieved the highest classification accuracy with ANN in classifying true emotional expressions from fake expressions in the alpha and beta bands with an average accuracy of 94.3% and 84.1%, respectively. Our results suggest combining DWT-EMD for future emotion studies and highlight the association of alpha and beta frequency bands with emotions.

Neural Decoding of EEG Signals with Machine Learning: A Systematic Review

“…In contrast, the studies that did not use this shared data set [ 65 , 218 , 219 , 225 , 226 , 227 , 228 , 229 , 230 ] all achieved higher classification performance by using SVM, KNN, and ANNs algorithms. For example, Seo et al [ 227 ] have compared different ML classifiers to classify boredom and non-boredom in 28 participants on the basis of the historical models of emotion and found that the KNN outperformed both RF and ANNs.…”

“… 60 channels Own database PCA SVM Accuracy = 71.7 [ 251 ] 2007 MWL 7 subj. 6 channels Own database AR RBF-SVM Accuracy = 70 [ 229 ] 2020 ER 28 subj. 64 channels Own database DWT/EMD ANNs KNN SVM Accuracy = 94.3 [ 297 ] 2016 ERP 52 subj.…”

Application of long short term memory algorithm in classification electroencephalogram

“…Từ khoá: dữ liệu hai chiều, dữ liệu ba chiều, điện não đồ, cảm xúc, mạng bộ nhớ dài ngắn hạn GIỚI THIỆU 1 Trong những năm gần đây, đã có rất nhiều các phương 2 pháp được sử dụng để trích xuất đặc trưng của tín 3 hiệu cảm xúc con người như là biểu hiện khuôn mặt 1 , 4 giọng nói 2,3 , nháy mắt 4 , hoặc sử dụng các tín hiệu 5 sinh lý. Khi so sánh các biểu hiện đã đề cập, chỉ có 6 biểu hiện sử dụng các tín hiệu sinh lý từ con người 7 là được các nhà nghiên cứu đánh giá cao về độ tin 8 cậy 5 , do việc sử dụng tín hiệu sinh lý đã được các nhà 9 nghiên cứu khẳng định rằng là khó có thể làm giả 6 .…”

Discriminating Fake and Real Smiles Using Electroencephalogram Signals With Convolutional Neural Networks

“…We are using this emotion model in the study. We selected images from the Geneva Affective Picture Database (GAPED) image dataset [13], [14], as stimuli. These images are designed to elicit certain kind of emotional response from the subjects.…”

“…The main contribution of this work lies in the use of an automated feature extraction technique via multiple convolutional neural networks (CNN). In [13], the features were extracted manually using the discrete wavelet transform (DWT) and empirical mode decomposition (EMD). In addition, we have validated the proposed CNN networks on the raw data using multiple other techniques like long short-term memory (LSTM) networks, shallow artificial neural network (ANN), and support vector machines (SVM).…”