Graph Reasoning-Based Emotion Recognition Network

Gao, Qinquan; Zeng, Hanxin; Li, Gen; Tong, Tong

doi:10.1109/access.2020.3048693

Cited by 18 publications

(7 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the CAER-S, we also compare our method to several recent state-of-the-art approaches. GRERN [22] utilized a multi-layer Graph Convolutional Fig. 5 The Fusion Module consists of two separate sub-networks, each network computes the fusion weights for face branch and context branch.…”

Section: Methodsmentioning

confidence: 99%

Global-local attention for emotion recognition

Nguyen

et al. 2021

Neural Comput & Applic

View full text Add to dashboard Cite

Human emotion recognition is an active research area in artificial intelligence and has made substantial progress over the past few years. Many recent works mainly focus on facial regions to infer human affection, while the surrounding context information is not effectively utilized. In this paper, we proposed a new deep network to effectively recognize human emotions using a novel global-local attention mechanism. Our network is designed to extract features from both facial and context regions independently, then learn them together using the attention module. In this way, both the facial and contextual information is used to infer human emotions, therefore enhancing the discrimination of the classifier. The intensive experiments show that our method surpasses the current state-of-the-art methods on recent emotion datasets by a fair margin. Qualitatively, our global-local attention module can extract more meaningful attention maps than previous methods. The source code and trained model of our network are available at https://github.com/minhnhatvt/glamor-net.

show abstract

Section: Methodsmentioning

confidence: 99%

Global-local attention for emotion recognition

Nguyen

et al. 2021

Neural Comput & Applic

View full text Add to dashboard Cite

show abstract

“…The accuracy of graph-based network on the Ekman-6 and VideoEmotion-8 data sets is 55.01% and 51.77%, respectively. It utilizes the semantic relationships of different regions based on the graph convolutional network [ 36 ] to improve performance. The results show that our method achieves the state-of-the-art results on both Ekman-6 and VideoEmotion-8 data sets.…”

Section: Methodsmentioning

confidence: 99%

Region Dual Attention-Based Video Emotion Recognition

Liu

Wang

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

To solve the emotional differences between different regions of the video frame and make use of the interrelationship between different regions, a region dual attention-based video emotion recognition method (RDAM) is proposed. RDAM takes as input video frame sequences and learns a discriminatory video emotion representation that can make full use of the emotional differences of different regions and the interrelationship between regions. Specifically, we construct two parallel attention modules: one is the regional location attention module, which generates a weight value for each feature region to identify the relative importance of different regions. Based on the weight, the emotion feature that can perceive the emotional sensitive region is generated. The other is the regional relationship attention module, which generates a region relation matrix that represents the interrelationship of different regions of a video frame. Based on the region relation matrix, the emotion feature that can perceive interrelationship between different regions is generated. The outputs of these two attention modules are fused to produce the emotional features of video frames. Then, the features of video frame sequences are fused by attention-based fusion network, and the final emotion feature of the video is produced. The experimental results on the video emotion recognition data sets show that the proposed method outperforms the other related works.

show abstract

“…Kernelized feature [ 26 ] and concept selection [ 27 ] studied frame relationships or regions of interest of emotion, which further improve the accuracy. Graph-based network [ 36 ] utilizes the semantic relationships of different regions based on the graph convolutional network to improve accuracy. Our previous work CAAN [ 37 ] only solves the difference of contained emotion information in different images.…”

Section: Methodsmentioning

confidence: 99%

“…Ekman (%) VideoEmotion-8 (%) Emotion in context [10] 51.8 50.6 Xu et al [33] 50.4 46.7 Kernelized feature [26] 54.4 49.7 Concept selection [27] 54.40 50.82 Graph-based network [36] 55.01 51.77 CAAN [37] 56.23 52.5 Ours 57.7 53.13…”

Section: Methodsmentioning

confidence: 99%

Hierarchical Attention‐Based Multimodal Fusion Network for Video Emotion Recognition

Liu

Wang

2021

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

The context, such as scenes and objects, plays an important role in video emotion recognition. The emotion recognition accuracy can be further improved when the context information is incorporated. Although previous research has considered the context information, the emotional clues contained in different images may be different, which is often ignored. To address the problem of emotion difference between different modes and different images, this paper proposes a hierarchical attention-based multimodal fusion network for video emotion recognition, which consists of a multimodal feature extraction module and a multimodal feature fusion module. The multimodal feature extraction module has three subnetworks used to extract features of facial, scene, and global images. Each subnetwork consists of two branches, where the first branch extracts the features of different modes, and the other branch generates the emotion score for each image. Features and emotion scores of all images in a modal are aggregated to generate the emotion feature of the modal. The other module takes multimodal features as input and generates the emotion score for each modal. Finally, features and emotion scores of multiple modes are aggregated, and the final emotion representation of the video will be produced. Experimental results show that our proposed method is effective on the emotion recognition dataset.

show abstract

Graph Reasoning-Based Emotion Recognition Network

Cited by 18 publications

References 34 publications

Global-local attention for emotion recognition

Global-local attention for emotion recognition

Region Dual Attention-Based Video Emotion Recognition

Hierarchical Attention‐Based Multimodal Fusion Network for Video Emotion Recognition

Contact Info

Product

Resources

About