Human Emotion Recognition: Review of Sensors and Methods

Dzedzickis, Andrius; Kaklauskas, Artūras; Bučinskas, Vytautas

doi:10.3390/s20030592

Cited by 299 publications

(133 citation statements)

References 147 publications

(174 reference statements)

Supporting

Mentioning

129

Contrasting

Unclassified

Order By: Relevance

“…Thus, emotion recognition is a critical factor for several domains such as human robot interaction, characterizing the level of interest on learning, measuring happiness and satisfaction, identifying the level of vigilance in road and safety, quantifying stress, and detecting patient's mental and physical states [1], [3]. Several methods have been proposed in the literature to evaluate emotions [4], [5], [6], [7]. Self-reporting may be the most straightforward approach to assess emotions and emotional behavior.…”

Section: Introductionmentioning

confidence: 99%

Discrimination of Genuine and Acted Emotional Expressions Using EEG Signal and Machine Learning

et al. 2020

View full text Add to dashboard Cite

We present here one of the first studies that attempt to differentiate between genuine and acted emotional expressions, using EEG data. We present the first EEG dataset (available here) with recordings of subjects with genuine and fake emotional expressions. We build our experimental paradigm for classification of smiles; genuine smiles, fake/acted smiles and neutral expression. We propose multiple methods to extract intrinsic features from three EEG emotional expressions; genuine, neutral, and fake/acted smile. We extracted EEG features using three time-frequency analysis methods: discrete wavelet transforms (DWT), empirical mode decomposition (EMD), and incorporating DWT into EMD (DWT-EMD) at three frequency bands. We then evaluated the proposed methods using several classifiers including, k-nearest neighbors (KNN), support vector machine (SVM), and artificial neural network (ANN). We carried out an experimental paradigm on 28-subjects underwent three types of emotional expressions, genuine, neutral and fake/acted. The results showed that incorporating DWT into EMD extracted more hidden features than sole DWT or sole EMD method. The power spectral feature extracted by DWT, EMD, and DWT-EMD showed different neural patterns across the three emotional expressions at all the frequency bands. We performed binary classification experiments and achieved acceptable accuracy reaching a maximum of 84% in all type of emotions, classifiers and bands using sole DWT or EMD. Meanwhile, a combination of DWT-EMD achieved the highest classification accuracy with ANN in classifying true emotional expressions from fake expressions in the alpha and beta bands with an average accuracy of 94.3% and 84.1%, respectively. Our results suggest combining DWT-EMD for future emotion studies and highlight the association of alpha and beta frequency bands with emotions.

show abstract

Section: Introductionmentioning

confidence: 99%

Discrimination of Genuine and Acted Emotional Expressions Using EEG Signal and Machine Learning

et al. 2020

View full text Add to dashboard Cite

show abstract

“…On the other hand, Abhang et al [ 74 ] point out that high beta waves are associated with significant stress, anxiety, high energy, and high arousal, and Dzedzickis, Kaklauskas, and Bucinskas [ 42 ] associated them to an alert or anxious state. According to these studies, it is plausible to find less arousal in the less demanding perceptual and relational “throwing” condition.…”

Section: Discussionmentioning

confidence: 99%

“…Beta values are high in situations of amusement. Dzedzickis, Kaklauskas, and Bucinskas [ 42 ], in their review on emotion recognition through EEG, show that beta waves appear associated to an alert or anxious state. Also, negative emotions are related to increased beta responses in humans [ 43 ].…”

Section: Introductionmentioning

confidence: 99%

Brain Activity during Different Throwing Games: EEG Exploratory Study

Monge

Rodríguez-Navarro

González-Calvo

et al. 2020

IJERPH

View full text Add to dashboard Cite

The purpose of this study is to explore the differences in brain activity in various types of throwing games by making encephalographic records. Three conditions of throwing games were compared looking for significant differences (simple throwing, throwing to a goal, and simultaneous throwing with another player). After signal processing, power spectral densities were compared through variance analysis (p ≤ 0.001). Significant differences were found especially in high-beta oscillations (22–30 Hz). “Goal” and “Simultaneous” throwing conditions show significantly higher values than those shown for throws without opponent. This can be explained by the higher demand for motor control and the higher arousal in competition situations. On the other hand, the high-beta records of the “Goal” condition are significantly higher than those of the “Simultaneous” throwing, which could be understood from the association of the beta waves with decision-making processes. These results support the difference in brain activity during similar games. This has several implications: opening up a path to study the effects of each specific game on brain activity and calling into question the transfer of research findings on animal play to all types of human play.

show abstract

“…Detailed information on the current state-of-the-art in a more generalized perspective, we refer the reader to the surveys [ 2 , 11 , 43 , 44 , 45 , 46 , 47 ] and references therein, where a comprehensive review of the latest work on ER using ML and physiological signals can be found, highlighting the main achievements, challenges, take-home messages, and possible future opportunities.…”

Section: State Of the Artmentioning

confidence: 99%

Emotion Assessment Using Feature Fusion and Decision Fusion Classification Based on Physiological Data: Are We There Yet?

Bota

Wang²,

Fred

et al. 2020

Sensors

View full text Add to dashboard Cite

Emotion recognition based on physiological data classification has been a topic of increasingly growing interest for more than a decade. However, there is a lack of systematic analysis in literature regarding the selection of classifiers to use, sensor modalities, features and range of expected accuracy, just to name a few limitations. In this work, we evaluate emotion in terms of low/high arousal and valence classification through Supervised Learning (SL), Decision Fusion (DF) and Feature Fusion (FF) techniques using multimodal physiological data, namely, Electrocardiography (ECG), Electrodermal Activity (EDA), Respiration (RESP), or Blood Volume Pulse (BVP). The main contribution of our work is a systematic study across five public datasets commonly used in the Emotion Recognition (ER) state-of-the-art, namely: (1) Classification performance analysis of ER benchmarking datasets in the arousal/valence space; (2) Summarising the ranges of the classification accuracy reported across the existing literature; (3) Characterising the results for diverse classifiers, sensor modalities and feature set combinations for ER using accuracy and F1-score; (4) Exploration of an extended feature set for each modality; (5) Systematic analysis of multimodal classification in DF and FF approaches. The experimental results showed that FF is the most competitive technique in terms of classification accuracy and computational complexity. We obtain superior or comparable results to those reported in the state-of-the-art for the selected datasets.

show abstract

Human Emotion Recognition: Review of Sensors and Methods

Cited by 299 publications

References 147 publications

Discrimination of Genuine and Acted Emotional Expressions Using EEG Signal and Machine Learning

Discrimination of Genuine and Acted Emotional Expressions Using EEG Signal and Machine Learning

Brain Activity during Different Throwing Games: EEG Exploratory Study

Emotion Assessment Using Feature Fusion and Decision Fusion Classification Based on Physiological Data: Are We There Yet?

Contact Info

Product

Resources

About