Exploiting emotional concepts for image emotion recognition

Yang, Hansen; Fan, Yangyu; Lv, Guoyun; Liu, Shiya; Guo, Zhe

doi:10.1007/s00371-022-02472-8

Cited by 14 publications

(4 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, physical signals belong to human external reactions, and humans can hide their emotional expressions. In addition, image-based emotion recognition methods do not reflect genuine human emotions but instead, replace humans in emotion recognition of images ( Yang et al, 2023 ). Therefore, the emotions conveyed by these physical signals cannot accurately capture the true thoughts of humans.…”

Section: Introductionmentioning

confidence: 99%

E-MFNN: an emotion-multimodal fusion neural network framework for emotion recognition

Guo,

Yang,

Lin

et al. 2024

PeerJ Computer Science

View full text Add to dashboard Cite

Emotional recognition is a pivotal research domain in computer and cognitive science. Recent advancements have led to various emotion recognition methods, leveraging data from diverse sources like speech, facial expressions, electroencephalogram (EEG), electrocardiogram, and eye tracking (ET). This article introduces a novel emotion recognition framework, primarily targeting the analysis of users’ psychological reactions and stimuli. It is important to note that the stimuli eliciting emotional responses are as critical as the responses themselves. Hence, our approach synergizes stimulus data with physical and physiological signals, pioneering a multimodal method for emotional cognition. Our proposed framework unites stimulus source data with physiological signals, aiming to enhance the accuracy and robustness of emotion recognition through data integration. We initiated an emotional cognition experiment to gather EEG and ET data alongside recording emotional responses. Building on this, we developed the Emotion-Multimodal Fusion Neural Network (E-MFNN), optimized for multimodal data fusion to process both stimulus and physiological data. We conducted extensive comparisons between our framework’s outcomes and those from existing models, also assessing various algorithmic approaches within our framework. This comparison underscores our framework’s efficacy in multimodal emotion recognition. The source code is publicly available at https://figshare.com/s/8833d837871c78542b29.

show abstract

Section: Introductionmentioning

confidence: 99%

E-MFNN: an emotion-multimodal fusion neural network framework for emotion recognition

Guo,

Yang,

Lin

et al. 2024

PeerJ Computer Science

View full text Add to dashboard Cite

show abstract

“…However, with the advent of advanced technology and artificial intelligence, a new era of emotional recognition has emerged. Real-Time Monitoring and Assessment Systems with Facial Landmark Estimation for Emotional Recognition employ sophisticated algorithms and machine learning techniques to analyze facial expressions, body language, and vocal cues, providing a more objective and accurate assessment of an individual's emotional state [5]. By precisely capturing and interpreting subtle facial cues, such as muscle movements and microexpressions, these systems can identify and classify emotions, including happiness, sadness, anger, surprise, and more, in real-time [6].…”

Section: Introductionmentioning

confidence: 99%

Real-Time Monitoring and Assessment System with Facial Landmark Estimation for Emotional Recognition in Work

Zhu

2023

IJRITCC

View full text Add to dashboard Cite

The Model for Monitoring and Regulating Emotional States in the Work Environment based on Neural Networks and Emotion Recognition Algorithms presents an innovative approach to enhancing employee well-being and productivity by leveraging advanced technologies. This paper on the development of a system that utilizes neural networks and emotion recognition algorithms to monitor and interpret emotional cues exhibited by individuals in real-time within the work environment. With the uses of novel Directional Marker Controlled Facial Landmark (DMCFL) Emotion recognition algorithms are employed to analyze facial expressions, speech patterns, physiological data, and text-based communication to infer the emotional state of employees. Neural networks are then utilized to process this data and provide more sophisticated emotion classification and prediction. The emotional states are classified with the integrated Regression Logistics Classifier (RLC) model for classification. The analysis of the findings expressed that the real-time monitoring enables employers and supervisors to gain insights into the emotional well-being of employees, identifying patterns and potential issues. The system facilitates feedback and regulation mechanisms, allowing for personalized interventions and support tailored to individual emotional needs.

show abstract

“…In today's society, student-centered teaching methods are becoming mainstream, and personalized and contextualized learning are receiving increasing attention [1][2][3][4][5]. Therefore, it is necessary for teachers to understand and adjust students' learning emotions in a timely manner to achieve optimal learning outcomes [6][7][8][9][10][11].…”

Section: Introductionmentioning

confidence: 99%

Emotion Recognition in Learning Scenes Supported by Smart Classroom and Its Application

Zhu,

Zheng,

et al. 2023

View full text Add to dashboard Cite

Emotion recognition technology is one of the important applications of artificial intelligence and machine learning in the field of education. By recognizing the emotions of students in learning scenes, teachers can better understand the learning status of students and provide them with personalized learning resources and help. Current emotion recognition methods are mainly based on static facial emotions, neglecting the temporal features of facial emotions, which may lead to inaccurate recognition results. In order to overcome these challenges, this study conducts research on emotion recognition and its application in learning scenes supported by smart classrooms. The Transformer encoder is used to extract the temporal features of student facial emotions based on learning scenes, i.e., the selfattention module of the encoder is used to extract the temporal features of facial emotions in learning scenes. Residual attention networks, Transformers, and non-local neural networks are used to extract facial emotion features from different perspectives and levels. The combination of Vision-Transformer (ViT) and NetVLAD enables the model to learn the features of data from multiple perspectives, thereby improving the generalization ability of the model. The experimental results verify the effectiveness of the constructed model.

show abstract

Exploiting emotional concepts for image emotion recognition

Cited by 14 publications

References 43 publications

E-MFNN: an emotion-multimodal fusion neural network framework for emotion recognition

E-MFNN: an emotion-multimodal fusion neural network framework for emotion recognition

Real-Time Monitoring and Assessment System with Facial Landmark Estimation for Emotional Recognition in Work

Emotion Recognition in Learning Scenes Supported by Smart Classroom and Its Application

Contact Info

Product

Resources

About