A 3D-convolutional neural network framework with ensemble learning techniques for multi-modal emotion recognition

Salama, Elham S.; El-Khoribi, Reda A.; Shoman, Mahmoud; Shalaby, Mohamed

doi:10.1016/j.eij.2020.07.005

Cited by 62 publications

(24 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Salama et al [25] used a three-dimensional Convolutional neural network (3D-CNN) as a deep learning technique. Six basic layers are used to extract features and train the model.…”

Section: Deep Learning Methods (1d 2d and 3d-cnn)mentioning

confidence: 99%

“…It is the input layer, followed by two 3D convolutional layers. After each 3D layer, there is a max-pooling layer and then the last layer, the feature extraction layer [25].…”

Section: Deep Learning Methods (1d 2d and 3d-cnn)mentioning

confidence: 99%

“…Then they built three methods of classifying emotions: the emotion recognition approach based on EEG signals, the face-based emotion recognition approach, and the fusion-based emotion recognition approach. In the first approach based on EEG signals, the team used 3D-CNN to obtain final predictions of EEG signals using the arousal-valence model [25]. Goshvarpour and Goshvarpour built a new approach to extract nonlinear features to classify emotions based on physiological brain signals (EEG).…”

Section: An Electroencephalogram (Eeg)mentioning

confidence: 99%

See 2 more Smart Citations

A Comprehensive Review for Emotion Detection Based on EEG Signals: Challenges, Applications, and Open Issues

Abdulrahman¹,

Baykara²

2021

View full text Add to dashboard Cite

Emotion classification based on physiological signals has become a hot topic in the past decade. Many studies have attempted to classify emotions using various techniques, to discover human emotions accurately. This study focused on listing the most recent studies that have classified emotions based on electroencephalogram (EEG) signals. This study also focused on solving the problems and challenges facing researchers in emotion classification and EEG applications used in several fields. The plan of this study is based on a strategy with three aspects within specific rules: The first aspect is the methods; we chose studies that included new methods to extract features. The second aspect is the data sets. We tried to choose a study that classified the same data set. The third aspect is applications; we have listed many applications of the EEG in several areas. We concluded from this study that detecting human emotions using the EEG signals is one of the most reliable and widely used methods of detecting emotions in the past few years. Also, we have noticed that the EEG can detect human emotions, especially in psychiatry, for example, for epileptic patients whose emotions cannot be extracted using traditional methods such as facial expressions and tone of voice.

show abstract

“…Salama et al [25] used a three-dimensional Convolutional neural network (3D-CNN) as a deep learning technique. Six basic layers are used to extract features and train the model.…”

Section: Deep Learning Methods (1d 2d and 3d-cnn)mentioning

confidence: 99%

“…It is the input layer, followed by two 3D convolutional layers. After each 3D layer, there is a max-pooling layer and then the last layer, the feature extraction layer [25].…”

Section: Deep Learning Methods (1d 2d and 3d-cnn)mentioning

confidence: 99%

Section: An Electroencephalogram (Eeg)mentioning

confidence: 99%

See 1 more Smart Citation

A Comprehensive Review for Emotion Detection Based on EEG Signals: Challenges, Applications, and Open Issues

Abdulrahman¹,

Baykara²

2021

View full text Add to dashboard Cite

show abstract

“…The valence can be positive to negative, and the arousal can be calm to excited [41]. This work would categorize the input into its variations of valence and arousal [42]. Early methods of extracting facial expressions had been developed manually by developers by developing algorithms for extracted functions.…”

Section: A Facial Expression Recognitionmentioning

confidence: 99%

Multimodal Emotion Recognition using Deep Learning

Abdullah

Ameen

Sadeeq

et al. 2021

JASTT

152

View full text Add to dashboard Cite

New research into human-computer interaction seeks to consider the consumer's emotional status to provide a seamless human-computer interface. This would make it possible for people to survive and be used in widespread fields, including education and medicine. Multiple techniques can be defined through human feelings, including expressions, facial images, physiological signs, and neuroimaging strategies. This paper presents a review of emotional recognition of multimodal signals using deep learning and comparing their applications based on current studies. Multimodal affective computing systems are studied alongside unimodal solutions as they offer higher accuracy of classification. Accuracy varies according to the number of emotions observed, features extracted, classification system and database consistency. Numerous theories on the methodology of emotional detection and recent emotional science address the following topics. This would encourage studies to understand better physiological signals of the current state of the science and its emotional awareness problems.

show abstract

“…Our proposed approach improved the emotion recognition performance when compared with other spatial or state-ofthe-art methods using the same DEAP dataset, as can be seen in Table I in Section II. Specifically, for 3D-based frameworks, a recent study of emotion recognition using DEAP dataset was conducted by Salama et al [46]. The authors proposed a 3D-CNN framework with ensemble learning and achieved recognition accuracies 96.13% and 96.79% for valence and arousal classes respectively in the subjectdependent case.…”

Section: F Comparison With State-of-the-art Modelsmentioning

confidence: 99%

EEG Classification by Factoring in Sensor Spatial Configuration

et al. 2021

View full text Add to dashboard Cite

Electroencephalography (EEG) serves as an effective diagnostic tool for mental disorders and neurological abnormalities. Enhanced analysis and classification of EEG signals can help improve performance in classifying the disorders and abnormalities. A new approach is examined here for enhancing EEG classification performance using a novel model of data representation that leverages knowledge of spatial layout of EEG sensors. An investigation of the performance of the proposed data representation model provides evidence of consistently higher classification accuracy of the proposed model compared with a model that ignores the sensor layout. The performance is assessed for models that represent the information content of the EEG signals in two different ways: a one-dimensional concatenation of the channels of the frequency bands and a proposed image-like two-dimensional representation of the EEG channel locations. The models are used in conjunction with different machine learning techniques. Performance of these models is examined on two tasks: social anxiety disorder classification, and emotion recognition using a dataset, DEAP, for emotion analysis using physiological signals. We hypothesize that the proposed two-dimensional model will significantly outperform the one-dimensional model and this is validated in our results as this model consistently yields 5-8% higher accuracy in all machine learning algorithms investigated. Among the algorithms investigated, Convolutional Neural Networks provide the best performance, far exceeding that of Support Vector Machine and k-Nearest Neighbors algorithms.

show abstract

A 3D-convolutional neural network framework with ensemble learning techniques for multi-modal emotion recognition

Cited by 62 publications

References 20 publications

A Comprehensive Review for Emotion Detection Based on EEG Signals: Challenges, Applications, and Open Issues

A Comprehensive Review for Emotion Detection Based on EEG Signals: Challenges, Applications, and Open Issues

Multimodal Emotion Recognition using Deep Learning

EEG Classification by Factoring in Sensor Spatial Configuration

Contact Info

Product

Resources

About