Physiology-based Recognition of Facial Micro-expressions using EEG and Identification of the Relevant Sensors by Emotion

Abstract

In this paper, we present a novel work about predicting the facial expressions from physiological signals of the brain. The main contributions of this paper are twofold. a) Investigation of the predictability of facial micro-expressions from EEG. b) Identification of the relevant features to the prediction. To reach our objectives, an experiment was conducted and we have proceeded in three steps: i) We recorded facial expressions and the corresponding EEG signals of participant while he/she is looking at pictures stimuli from the IAPS (International Affective Picture System). ii) We fed machine learning algorithms with timedomain and frequency-domain features of one second EEG signals with also the corresponding facial expression data as ground truth in the training phase. iii) Using the trained classifiers, we predict facial emotional reactions without the need to a camera. Our method leads us to very promising results since we have reached high accuracy. It also provides an additional important result by locating which electrodes can be used to characterize specific emotion. This system will be particularly useful to evaluate emotional reactions in virtual reality environments where the user is wearing VR headset that hides the face and makes the traditional webcam facial expression detectors obsolete. 130Benlamine, M., Chaouachi, M., Frasson, C. and Dufresne, A. Physiology-based Recognition of Facial Micro-expressions using EEG and Identification of the Relevant Sensors by Emotion.

Machine Learning Research Trends in Africa: A 30 Years Overview with Bibliometric Analysis Review

“…Mbona and Eloff [ 126 ] designed a semi-supervised machine learning approach to detect zero-day (new unknown) intrusion attacks based on the law of anomalous numbers to identify significant network features that effectively show anomalous behaviour. Similarly, Benlamine et al [ 127 ], used a machine learning model to evaluate emotional reactions in virtual reality environments where the face is hidden in a virtual reality headset, making facial expression detection using a webcam impossible. Several machine learning techniques have been used to identify and classify spam e-mails [ 128 ].…”

Using Facial Micro-Expressions in Combination With EEG and Physiological Signals for Emotion Recognition

“…Some studies focused on the relationship between behavioral responses and physiological changes in multimodal emotion recognition. For example, Benlamine et al ( 2016 ) and Raheel et al ( 2019 ) used EEG signals to recognize facial micro-expressions. Hassouneh et al ( 2020 ) used single-modality strategies for recognizing emotion in physically disabled people or people with autism using EEG and facial data.…”

Cognitive Workload Assessment of Aircraft Pilots

“…To extract the CW from the EEG headset, this research uses a third-party software called Mentor [10]. This software is a module from the NCO software, a proprietary software of the BMU lab that represents the convergence of multiple years of research into one extensive program called NCO [11]. The program uses machine learning models to extract the EEG signals, interpret them and transform them into a readable cognitive workload score ranging from 0 to 100 [10].…”