Multimodal fusion framework: A multiresolution approach for emotion classification and recognition from physiological signals

Verma, Gyanendra K.; Tiwary, Uma Shanker

doi:10.1016/j.neuroimage.2013.11.007

Cited by 278 publications

(124 citation statements)

References 32 publications

Supporting

Mentioning

109

Contrasting

Unclassified

Order By: Relevance

“…It also provided a comprehensive background for multi-modal emotional state and proposed a frame work and discussed the feasibility and utility of using the multi-modal approach. In [8], the researchers used multimodal physiological signals EEG, Galvanic skin response, blood volume pressure, respiration pattern, skin temperature, electromyogram, and electrooculogram to classify and predict depression. They proved the potential of the multimodal approach and achieved 85.46% accuracy using support vector machine (SVM).…”

Section: Design Innovation (Experimental) Papermentioning

confidence: 99%

See 1 more Smart Citation

Review and Classification of Emotion Recognition Based on EEG Brain-Computer Interface System Research: A Systematic Review

et al. 2017

View full text Add to dashboard Cite

Abstract:Recent developments and studies in brain-computer interface (BCI) technologies have facilitated emotion detection and classification. Many BCI studies have sought to investigate, detect, and recognize participants' emotional affective states. The applied domains for these studies are varied, and include such fields as communication, education, entertainment, and medicine. To understand trends in electroencephalography (EEG)-based emotion recognition system research and to provide practitioners and researchers with insights into and future directions for emotion recognition systems, this study set out to review published articles on emotion detection, recognition, and classification. The study also reviews current and future trends and discusses how these trends may impact researchers and practitioners alike. We reviewed 285 articles, of which 160 were refereed journal articles that were published since the inception of affective computing research. The articles were classified based on a scheme consisting of two categories: research orientation and domains/applications. Our results show considerable growth of EEG-based emotion detection journal publications. This growth reflects an increased research interest in EEG-based emotion detection as a salient and legitimate research area. Such factors as the proliferation of wireless EEG devices, advances in computational intelligence techniques, and machine learning spurred this growth.

show abstract

Section: Design Innovation (Experimental) Papermentioning

confidence: 99%

“…[3,7,8,[10][11][12][13][14][15][16][22][23][24][25] Different approaches have been proposed, and several research groups have developed EEG-based BCI systems that aim to detect affective state. Examples of non-medical applications are hereafter identified.…”

Section: Domain Description Referencesmentioning

confidence: 99%

Review and Classification of Emotion Recognition Based on EEG Brain-Computer Interface System Research: A Systematic Review

et al. 2017

View full text Add to dashboard Cite

show abstract

“…Attempts have already been made in developing cooperative strategies in supervised classification with ensemble models [34], or by considering multi-scaled sliding windows for binary classification [35]. Cooperative strategies have also been used to perform fusion of multimodal stimuli, by using either early (i. e., features) or late (i. e., decisions) fusion techniques [26], [36], [37], [38].…”

Section: Related Workmentioning

confidence: 99%

Continuous Estimation of Emotions in Speech by Dynamic Cooperative Speaker Models

Mencattini

Martinelli

Ringeval

et al. 2017

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

Abstract-Automatic emotion recognition from speech has been recently focused on the prediction of time-continuous dimensions (e.g., arousal and valence) of spontaneous and realistic expressions of emotion, as found in real-life interactions. However, the automatic prediction of such emotions poses several challenges, such as the subjectivity found in the definition of a gold standard from a pool of raters and the issue of data scarcity in training models. In this work, we introduce a novel emotion recognition system, based on ensemble of single-speaker-regression-models (SSRMs). The estimation of emotion is provided by combining a subset of the initial pool of SSRMs selecting those that are most concordance among them. The proposed approach allows the addition or removal of speakers from the ensemble without the necessity to re-build the entire machine learning system. The simplicity of this aggregation strategy, coupled with the flexibility assured by the modular architecture, and the promising results obtained on the RECOLA database highlight the potential implications of the proposed method in a real-life scenario and in particular in WEB-based applications.

show abstract

“…Verma and Tiwary [6] reported the use of the EEG signals for emotion recognition using the power spectral density as features and the Support Vector Machines (SVM) and the kNearest Neighbors (KNN) as classifiers. Yoon and Chung [7] introduced a new emotion recognition method using the EEG signals.…”

Section: Related Workmentioning

confidence: 99%

EEG-Based Emotion Recognition using 3D Convolutional Neural Networks

Salama¹,

El-Khoribi²,

Shoman³

et al. 2018

ijacsa

151

View full text Add to dashboard Cite

Abstract-Emotion recognition is a crucial problem in Human-Computer Interaction (HCI).Various techniques were applied to enhance the robustness of the emotion recognition systems using electroencephalogram (EEG) signals especially the problem of spatiotemporal features learning. In this paper, a novel EEG-based emotion recognition approach is proposed. In this approach, the use of the 3-Dimensional Convolutional Neural Networks (3D-CNN) is investigated using a multi-channel EEG data for emotion recognition. A data augmentation phase is developed to enhance the performance of the proposed 3D-CNN approach. And, a 3D data representation is formulated from the multi-channel EEG signals, which is used as data input for the proposed 3D-CNN model. Extensive experimental works are conducted using the DEAP (Dataset of Emotion Analysis using the EEG and Physiological and Video Signals) data. It is found that the proposed method is able to achieve recognition accuracies 87.44% and 88.49% for valence and arousal classes respectively, which is outperforming the state of the art methods.

show abstract

Multimodal fusion framework: A multiresolution approach for emotion classification and recognition from physiological signals

Cited by 278 publications

References 32 publications

Review and Classification of Emotion Recognition Based on EEG Brain-Computer Interface System Research: A Systematic Review

Review and Classification of Emotion Recognition Based on EEG Brain-Computer Interface System Research: A Systematic Review

Continuous Estimation of Emotions in Speech by Dynamic Cooperative Speaker Models

EEG-Based Emotion Recognition using 3D Convolutional Neural Networks

Contact Info

Product

Resources

About