Comparative analysis of physiological signals and electroencephalogram (EEG) for multimodal emotion recognition using generative models

Torres-Valencia, Cristian A.; Garcia-Arias, Hernan F.; López, Mariano Jiménez; Gutiérrez, Álvaro Ángel Orozco

doi:10.1109/stsiva.2014.7010181

Cited by 45 publications

(28 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The level feature fusion is to combine all the modalities before the training stage [38]. Thus, a simple concatenation was applied to all the extracted features.…”

Section: Resultsmentioning

confidence: 99%

Emotion Classification in Arousal Valence Model using MAHNOB-HCI Database

Wiem¹,

Lachiri²

2017

ijacsa

View full text Add to dashboard Cite

Abstract-Emotion recognition from physiological signals attracted the attention of researchers from different disciplines, such as affective computing, cognitive science and psychology. This paper aims to classify emotional statements using peripheral physiological signals based on arousal-valence evaluation. These signals are the Electrocardiogram, Respiration Volume, Skin Temperature and Galvanic Skin Response. We explored the signals collected in the MAHNOB-HCI multimodal tagging database. We defined the emotion into three different ways: two and three classes using 1-9 discrete self-rating scales and another model using 9 emotional keywords to establish the three defined areas in arousal-valence dimensions. To perform the accuracies, we began by removing the artefacts and noise from the signals, and then we extracted 169 features. We finished by classifying the emotional states using the support vector machine. The obtained results showed that the electrocardiogram and respiration volume were the most relevant signals for human's feeling recognition task. Moreover, the obtained accuracies were promising comparing to recent related works for each of the three establishments of emotion modeling.

show abstract

“…The level feature fusion is to combine all the modalities before the training stage [38]. Thus, a simple concatenation was applied to all the extracted features.…”

Section: Resultsmentioning

confidence: 99%

Emotion Classification in Arousal Valence Model using MAHNOB-HCI Database

Wiem¹,

Lachiri²

2017

ijacsa

View full text Add to dashboard Cite

show abstract

“…Such approaches include Naive Bayes (NB) [40], [41], linear discriminant analysis (LDA) [42], and support vector machine (SVM) [43], [44], [45]. A summary can be found in Table I. A number of studies have sought to model temporal information within EEG signals, using hidden Markov models [46], Gaussian Process models [47], continuous conditional random fields [48], and long short-term memory (LSTM) neural networks [49]. Such temporal treatment, however, is rare for other physiological data.…”

Section: A Unimodal Heartbeat and Temporal Modelsmentioning

confidence: 99%

A Bayesian Deep Learning Framework for End-To-End Prediction of Emotion From Heartbeat

Harper

Southern

2022

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

Automatic prediction of emotion promises to revolutionise human-computer interaction. Recent trends involve fusion of multiple modalities − audio, visual, and physiological − to classify emotional state. However, practical considerations 'in the wild' limit collection of this physiological data to commoditised heartbeat sensors. Furthermore, real-world applications often require some measure of uncertainty over model output. We present here an end-to-end deep learning model for classifying emotional valence from unimodal heartbeat data. We further propose a Bayesian framework for modelling uncertainty over valence predictions, and describe a procedure for tuning output according to varying demands on confidence. We benchmarked our framework against two established datasets within the field and achieved peak classification accuracy of 90%. These results lay the foundation for applications of affective computing in realworld domains such as healthcare, where a high premium is placed on non-invasive collection of data, and predictive certainty.

show abstract

“…These findings have reshaped scientific understanding of EEG signals and inspired following works to analyze them directly instead of performing the feature extraction step. For example, [30] classified the EEG preprocessed signals using the Hidden Markov Model (HMM). Likewise, feature learning was performed by feeding the raw channel data to the Deep Belief Network (DBN) [29].…”

Section: Related Work On Deap Datasetmentioning

confidence: 99%

Unsupervised Learning in Reservoir Computing for EEG-Based Emotion Recognition

Fourati

Ammar

Medina

et al. 2022

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

In real-world applications such as emotion recognition from recorded brain activity, data are captured from electrodes over time. These signals constitute a multidimensional time series. In this paper, Echo State Network (ESN), a recurrent neural network with a great success in time series prediction and classification, is optimized with different neural plasticity rules for classification of emotions based on electroencephalogram (EEG) time series. Actually, the neural plasticity rules are a kind of unsupervised learning adapted for the reservoir, i.e. the hidden layer of ESN. More specifically, an investigation of Oja's rule, BCM rule and gaussian intrinsic plasticity rule was carried out in the context of EEG-based emotion recognition. The study, also, includes a comparison of the offline and online training of the ESN. When testing on the well-known affective benchmark "DEAP dataset" which contains EEG signals from 32 subjects, we find that pretraining ESN with gaussian intrinsic plasticity enhanced the classification accuracy and outperformed the results achieved with an ESN pretrained with synaptic plasticity. Four classification problems were conducted in which the system complexity is increased and the discrimination is more challenging, i.e. inter-subject emotion discrimination. Our proposed method achieves higher performance over the state of the art methods.

show abstract

Comparative analysis of physiological signals and electroencephalogram (EEG) for multimodal emotion recognition using generative models

Cited by 45 publications

References 16 publications

Emotion Classification in Arousal Valence Model using MAHNOB-HCI Database

Emotion Classification in Arousal Valence Model using MAHNOB-HCI Database

A Bayesian Deep Learning Framework for End-To-End Prediction of Emotion From Heartbeat

Unsupervised Learning in Reservoir Computing for EEG-Based Emotion Recognition

Contact Info

Product

Resources

About