Towards emotion recognition for virtual environments: an evaluation of eeg features on benchmark dataset

Menezes, Maria Luiza Recena; Samara, Anas; Galway, Leo; Sant'Anna, Anita Pinheiro; Verikas, Antanas; Alonso‐Fernandez, Fernando; Wang, H.; Bond, Raymond

doi:10.1007/s00779-017-1072-7

Cited by 82 publications

(55 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The literature reveals that the vast majority of the existing work related to EEG-based emotion analysis has utilized the arousal and valence scales to describe different emotional states [ 4 , 23 , 24 ]. Therefore, in this study, we focus on the values of the valence and arousal scales to quantify and describe the various emotional states.…”

Section: Methodsmentioning

confidence: 99%

“…Figure 1 a illustrates the emotion classes defined based on the 1D-2CLS. The one-dimensional three-class labeling scheme (1D-3CLS) [ 4 , 28 , 29 , 30 , 31 ]: This emotion labeling scheme utilizes the arousal and valence scales independently to define three emotion classes for each scale. In particular, using the arousal scale, a trial is assigned to the high arousal (HA) emotion class, the neutral emotion class or the low arousal (LA) emotion class depending on whether the associated arousal value is within the interval [6.5–9], (3.5–6.5) or [1–3.5], respectively.…”

Section: Methodsmentioning

confidence: 99%

“…For example, in the medical domain, discerning the patient’s emotional state can provide caregivers with an indicator about the patient’s mental and physical status [ 1 , 2 ] and the progress of the recovery process [ 3 ]. In the education domain, characterizing the students’ emotional state in terms of the level of interest and active participation in the learning process is crucial to achieve effective knowledge transfer and development systems [ 4 ]. In the road- and aviation-safety domain, identifying the level of the boredom and vigilance emotional states of the driver/pilot can contribute to reducing the rate of accidents [ 5 ].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

EEG-Based Emotion Recognition Using Quadratic Time-Frequency Distribution

Alazrai

Homoud

Alwanni

et al. 2018

Sensors

View full text Add to dashboard Cite

Accurate recognition and understating of human emotions is an essential skill that can improve the collaboration between humans and machines. In this vein, electroencephalogram (EEG)-based emotion recognition is considered an active research field with challenging issues regarding the analyses of the nonstationary EEG signals and the extraction of salient features that can be used to achieve accurate emotion recognition. In this paper, an EEG-based emotion recognition approach with a novel time-frequency feature extraction technique is presented. In particular, a quadratic time-frequency distribution (QTFD) is employed to construct a high resolution time-frequency representation of the EEG signals and capture the spectral variations of the EEG signals over time. To reduce the dimensionality of the constructed QTFD-based representation, a set of 13 time- and frequency-domain features is extended to the joint time-frequency-domain and employed to quantify the QTFD-based time-frequency representation of the EEG signals. Moreover, to describe different emotion classes, we have utilized the 2D arousal-valence plane to develop four emotion labeling schemes of the EEG signals, such that each emotion labeling scheme defines a set of emotion classes. The extracted time-frequency features are used to construct a set of subject-specific support vector machine classifiers to classify the EEG signals of each subject into the different emotion classes that are defined using each of the four emotion labeling schemes. The performance of the proposed approach is evaluated using a publicly available EEG dataset, namely the DEAPdataset. Moreover, we design three performance evaluation analyses, namely the channel-based analysis, feature-based analysis and neutral class exclusion analysis, to quantify the effects of utilizing different groups of EEG channels that cover various regions in the brain, reducing the dimensionality of the extracted time-frequency features and excluding the EEG signals that correspond to the neutral class, on the capability of the proposed approach to discriminate between different emotion classes. The results reported in the current study demonstrate the efficacy of the proposed QTFD-based approach in recognizing different emotion classes. In particular, the average classification accuracies obtained in differentiating between the various emotion classes defined using each of the four emotion labeling schemes are within the range of 73.8%–86.2%. Moreover, the emotion classification accuracies achieved by our proposed approach are higher than the results reported in several existing state-of-the-art EEG-based emotion recognition studies.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

EEG-Based Emotion Recognition Using Quadratic Time-Frequency Distribution

Alazrai

Homoud

Alwanni

et al. 2018

Sensors

View full text Add to dashboard Cite

show abstract

“…Their evaluation of the results on the DEAP dataset shows that the performance of the proposed method using the SVM classifier (average of 55%) is more robust and successful when the number of selected features is between 80 and 125. Menezes et al, (2017) extracted some limited features from time and frequency domains and applied SVM method to classify each emotion dimension (Arousal and Valence), into two and three-class (Bipartition and Tripartition). This model has been tested on DEAP dataset.…”

Section: Accepted Manuscriptmentioning

confidence: 99%

Evolutionary computation algorithms for feature selection of EEG-based emotion recognition using mobile sensors

Nakisa

Rastgoo

Tjondronegoro

et al. 2018

Expert Systems with Applications

194

View full text Add to dashboard Cite

There is currently no standard or widely accepted subset of features to effectively classify different emotions based on electroencephalogram (EEG) signals. While combining all possible EEG features may improve the classification performance, it can lead to high dimensionality and worse performance due to redundancy and inefficiency. To solve the high-dimensionality problem, this paper proposes a new framework to automatically search for the optimal subset of EEG features using evolutionary computation (EC) algorithms. The proposed framework has been extensively evaluated using two public datasets (MAHNOB, DEAP) and a new dataset acquired with a mobile EEG sensor. The results confirm that EC algorithms can effectively support feature selection to identify the best EEG features and the best channels to maximize performance over a four-quadrant emotion classification problem. These findings are significant for informing future development of EEG-based emotion classification because low-cost mobile EEG sensors with fewer electrodes are becoming popular for many new applications.

show abstract

“…The current approaches to emotion recognition involve acquisition of sensory data from different modalities including facial expression, heart rate, galvanic skin response and brainwaves [3,4]. Brainwave data or electroencephalography (EEG) is typically the method of choice for multi-class emotion classification due to its richer set of data acquired through the multiple electrodes located on the skull of the user.…”

Section: Introductionmentioning

confidence: 99%

Augmenting EEG with Inertial Sensing for Improved 4-Class Subject-Independent Emotion Classification in Virtual Reality

Teo¹,

Suhaimi²,

Mountstephens³

2019

Proceedings of the Proceedings of the 1st International Conference on Informatics, Engineering, Science and Technology, INCITES

View full text Add to dashboard Cite

This investigation reports on the promising results obtained from the novel use of inertial sensing data for augmenting electroencephalography (EEG)-based subjectindependent classification of emotions generated by virtual reality stimuli in four classes. 31 users were shown various virtual reality scenes to elicit responses in the four-quadrant emotional space according to Russell's Circumplex Model of Affect. Prior studies in emotion classification can be broadly grouped according to two main delineations of investigative methods: (1) whether the classification is binary (i.e. two-class classification) or otherwise (e.g. three-class, four-class classification or more) and (2) whether the training and testing occurs within the same participant (also known as subject-dependent classification) or across different participants (also known as subject-independent classification). Due to the significantly higher level of difficulty in conducting ternary/quaternary, subject-independent classification, the large majority of emotion modeling studies that report high accuracy rates adopts the binary, subject-dependent approach to classification. However, this study attempts the more challenging four-class classification, subject-independent classification. The EEG signals, accelerometer, and gyroscopic data were acquired through a wearable brain-computer interface device called Muse. Raw as well as power spectral density features were extracted from the EEG signals, and together with the first known use of inertial sensing data for emotion classification, were used as input to a deep neural network. Classification results show that without inertial sensing data, inter-subject classification was indeed highly challenging even for a deep learning system with only slightly better than random for 4-class classification at 26-27%. However, the augmentation of inertial sensing data improved the classification accuracy to 40-47%. As such, this work demonstrates the potential of using inertial sensing as an additional modality for emotion modeling.

show abstract

Towards emotion recognition for virtual environments: an evaluation of eeg features on benchmark dataset

Cited by 82 publications

References 35 publications

EEG-Based Emotion Recognition Using Quadratic Time-Frequency Distribution

EEG-Based Emotion Recognition Using Quadratic Time-Frequency Distribution

Evolutionary computation algorithms for feature selection of EEG-based emotion recognition using mobile sensors

Augmenting EEG with Inertial Sensing for Improved 4-Class Subject-Independent Emotion Classification in Virtual Reality

Contact Info

Product

Resources

About