Feature Extraction for Emotion Recognition and Modelling Using Neurophysiological Data

Abstract

The ubiquitous computing paradigm is becoming a reality; we are reaching a level of automation and computing in which people and devices interact seamlessly. However, one of the main challenges is the difficulty users have in interacting with these increasingly complex systems. Ultimately, endowing machines with the ability to perceive users emotions will enable a more intuitive and reliable interaction. Consequently, using the electroencephalogram (EEG) as a bio-signal sensor, the affective state of a user can be modelled and subsequently utilised in order to achieve a system that can recognise and react to the users emotions. In this context, this paper investigates feature vector generation from EEG signals for the purpose of affective state modelling based on Russells Circumplex Model. Investigations are presented that aim to provide the foundation for future work in modelling user affect and interaction experiences through exploitation of different input modalities. The DEAP dataset was used within this work, along with a Support Vector Machine, which yielded reasonable classification accuracies for Valence and Arousal using feature vectors based on statistical measurements and band power from the α, β, δ, and θ waves and High Order Crossing of the EEG signal.

A Review of Emotion Recognition Using Physiological Signals

“…The work of [ 37 , 102 , 103 , 141 ] all have ordinary performance respectively. This calls for improvement in applying an objective method for selecting a minimal or optimal feature subset, rather than ad hoc selected features.…”

“…This calls for improvement in applying an objective method for selecting a minimal or optimal feature subset, rather than ad hoc selected features. In the study of [ 141 ] (Ref No. 2 in Figure 18 ), the authors investigated feature vector generation from EEG signals, where only statistical features were considered through exhaustive test.…”

The Design of CNN Architectures for Optimal Six Basic Emotion Classification Using Multiple Physiological Signals

“…Much research has been conducted on emotional analysis [9,20,[58][59][60], and this research was based on a two-dimensional emotion model or a few discrete emotion models. In addition, emotion classification was performed with respiration signals and ECG signals [30,31,61], and multi-signal classification using signals such as EEG, GSR, EMG, and SKT, which are less relevant to respiration, has been used [10,62]. However, the dimension emotion model actually has a difference in feeling between the six basic emotions in daily life.…”

“…Emotion classification through defined emotions and measured signals is performed in various ways, ranging from traditional statistical methods and machine learning (linear regression, logistic regression, hidden Markov model, naïve Bayes classification, support vector machine, and Decision Tree) to the latest technique, deep learning. In particular, the support vector machine (SVM) is a widely used method for emotion classification [9][10][11]. Torres-Valencia et al [12] classified two-dimensional emotions by using HMM, and studies using the C4.5 Decision Tree [13], K-nearest neighbor (KNN) [14,15], and Linear Discriminant Analysis (LDA) [16] have been reported.…”

Design for an Art Therapy Robot: An Explorative Review of the Theoretical Foundations for Engaging in Emotional and Creative Painting with a Robot

“…Additionally, in the case where a person cannot or does not wish to make art themselves, the robot might have to rely on signals other than in the painting to infer their emotions. Various sensors, from cameras and microphones to electrodermal activity sensors, and brain machine interfaces, can be used to infer emotions, from signals such as facial expressions and speech [146][147][148].…”