Human emotion recognition using deep belief network architecture

Hassan, Mohammad Mehedi; Alam, Md. Golam Rabiul; Uddin, Md. Zia; Huda, Shamsul; Almogren, Ahmad; Fortino, Giancarlo

doi:10.1016/j.inffus.2018.10.009

Cited by 231 publications

(113 citation statements)

References 40 publications

Supporting

Mentioning

111

Contrasting

Unclassified

Order By: Relevance

“…Kim et al [15] obtained 110 features from four bio-signals (ECG, respiration, skin conductivity, and EMG), and selected hand-crafted features for different subjects and emotions among these features to represent emotions well. Hassan et al used electro-dermal activity (EDA), PPG, and EMG signals to extract statistical features from the PSD of amplitude versus occurrence distribution [30].…”

Section: Hand-crafted Features For Emotion Recognitionmentioning

confidence: 99%

Fast Emotion Recognition Based on Single Pulse PPG Signal with Convolutional Neural Network

Lee

Pae

et al. 2019

Applied Sciences

View full text Add to dashboard Cite

Physiological signals contain considerable information regarding emotions. This paper investigated the ability of photoplethysmogram (PPG) signals to recognize emotion, adopting a two-dimensional emotion model based on valence and arousal to represent human feelings. The main purpose was to recognize short term emotion using a single PPG signal pulse. We used a one-dimensional convolutional neural network (1D CNN) to extract PPG signal features to classify the valence and arousal. We split the PPG signal into a single 1.1 s pulse and normalized it for input to the neural network based on the personal maximum and minimum values. We chose the dataset for emotion analysis using physiological (DEAP) signals for the experiment and tested the 1D CNN as a binary classification (high or low valence and arousal), achieving the short-term emotion recognition of 1.1 s with 75.3% and 76.2% valence and arousal accuracies, respectively, on the DEAP data.

show abstract

Section: Hand-crafted Features For Emotion Recognitionmentioning

confidence: 99%

Fast Emotion Recognition Based on Single Pulse PPG Signal with Convolutional Neural Network

Lee

Pae

et al. 2019

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Hoy en día existen muchos trabajos de investigación enfocados al área de reconocimiento de emociones utilizando modalidades como la voz y el rostro para la extracción de características que han logrado crear modelos de alta precisión utilizando técnicas de ML [6] y DL [7] con el uso de redes de creencia profunda (BLN). Además, se ha observado un gran avance en la minería de opiniones y la clasificación de emociones en texto utilizando técnicas de DL [8].…”

Section: Trabajos Relacionadosunclassified

“…Ramón Zatarain Cabada, María Lucia Barrón Estrada, Héctor Manuel Cárdenas LópezResearch in Computing Science 148(7), 2019 ISSN 1870-4069…”

unclassified

Reconocimiento multimodal de emociones orientadas al aprendizaje

Zataraín-Cabada¹,

Barrón-Estrada²,

Lopez³

2019

RCS

View full text Add to dashboard Cite

Resumen. Las emociones centradas en el aprendizaje tienen un rol significativo en el proceso pedagógico del estudiante. Por esta razón, es relevante en los ambientes de aprendizaje virtual tomar en consideración no solo los aspectos cognitivos del estudiante si no también los aspectos afectivos. Algunos métodos como el auto reporte, observacional y las imágenes de actividad neuronal, han sido usados extensivamente para medir emociones. Sin embargo, hoy en día otros métodos de análisis de sentimientos y reconocimiento facial usando inteligencia artificial han demostrado ser una mejor alternativa para el reconocimiento automatizado de afecto. Los resultados son superiores cuando se combinan diferentes métodos para combinar el reconocimiento de expresiones faciales, expresiones textuales, aplicadas en ambientes de aprendizaje. Para la implementación de reconocedores, utilizamos nuestro propio corpus para la clasificación que emplea técnicas de aprendizaje profundo. Evaluamos la eficiencia de tres métodos de fusión comparándolas contra métodos unimodales de reconocimiento de emociones. La mejora de uno de los métodos (Representación basada en embebidos) de fusión fue de 4% (precisión de 86.20% y perdida 1.35%). Los resultados son muy prometedores y pensamos que serán mejor con tres o más reconocedores participando en el sistema multimodal.Palabras clave: reconocimiento de emociones, reconocimiento multimodal, método de fusión.Abstract. Learning-centered emotions have a significant role in the pedagogical process of a student. For this reason, it is relevant that virtual learning environments take into account not only the cognitive aspects of the student but also the affective ones. Some approaches such as self-report, observational, and neuroimaging have been used extensively to measure emotions. However, today other methods of artificial intelligence such as the recognition of facial expressions and sentimental analysis have proven to be a better alternative in the automatic recognition of affect, and the results are superior when we combine and integrate several modes of recognition. In this work, we present three different methods to combine the recognition of facial expressions and textual expressions, applied to virtual learning environments. For the implementation of recognizers, we use our own corpora with classifiers that employ deep learning techniques. We evaluate the efficiency of the three fusion methods by comparing them against unimodal methods of emotion recognition. The improvement of one 153

show abstract

“…For example, the current number of IoT devices will rapidly increase from 15 billion to 50 billion by 2020 (according to CISCO), while the number of sensors will increase to as high as 1 trillion by 2030 (according to HP Labs). In emotion analysis applications [22], we need multi-modal data for the emotion recognition, such application consumes lots of sensors for acquiring emotion-related data such as facial expression, voice and other physical data. The sustainability of such systems becomes a necessity.…”

Section: Introductionmentioning

confidence: 99%

A Sustainable Multi-Modal Multi-Layer Emotion-Aware Service at the Edge

Fortino

et al. 2022

IEEE Trans. Sustain. Comput.

Self Cite

View full text Add to dashboard Cite

Limited by the computational capabilities and battery energy of terminal devices and network bandwidth, emotion recognition tasks fail to achieve good interactive experience for users. The intolerable latency for users also seriously restricts the popularization of emotion recognition applications in the edge environments such as fatigue detection in auto-driving. The development of edge computing provides a more sustainable solution for this problem. Based on edge computing, this article proposes a multi-modal multi-layer emotion-aware service (MULTI-EASE) architecture that considers user's facial expression and voice as a multi-modal data source of emotion recognition, and employs the intelligent terminal, edge server and cloud as multi-layer execution environment. By analyzing the average delay of each task and the average energy consumption at the mobile device, we formulate a delay-constrained energy minimization problem and perform a task scheduling policy between multiple layers to reduce the end-to-end delay and energy consumption by using an edge-based approach, further to improve the users' emotion interactive experience and achieve energy saving in edge computing. Finally, a prototype system is also implemented to validate the architecture of MULTI-EASE, the experimental results show that MULTI-EASE is a sustainable and efficient platform for emotion analysis applications, and also provide a valuable reference for dynamic task scheduling under MULTI-EASE architecture.

show abstract

Human emotion recognition using deep belief network architecture

Cited by 231 publications

References 40 publications

Fast Emotion Recognition Based on Single Pulse PPG Signal with Convolutional Neural Network

Fast Emotion Recognition Based on Single Pulse PPG Signal with Convolutional Neural Network

Reconocimiento multimodal de emociones orientadas al aprendizaje

A Sustainable Multi-Modal Multi-Layer Emotion-Aware Service at the Edge

Contact Info

Product

Resources

About