A Multi-Attentive Pyramidal Model for Visual Sentiment Analysis

He, Xiaohao; Zhang, Huijun; Li, Ningyun; Ling, Feng; Zheng, Feng

doi:10.1109/ijcnn.2019.8852317

Cited by 20 publications

(10 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zhang et al [132] modeled the correlation between object semantics in different image regions to infer the image sentiment based on Bayesian networks. A multi-attentive pyramidal model is proposed in [133] to extract local features at various scales, and then, a selfattention mechanism is employed to mine the relations between features of different regions.…”

Section: Learning-based Methodsmentioning

confidence: 99%

Affective Image Content Analysis: Two Decades Review and New Perspectives

Zhao

Yao

Yang

et al. 2022

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Images can convey rich semantics and induce various emotions in viewers. Recently, with the rapid advancement of emotional intelligence and the explosive growth of visual data, extensive research efforts have been dedicated to affective image content analysis (AICA). In this survey, we will comprehensively review the development of AICA in the recent two decades, especially focusing on the state-of-the-art methods with respect to three main challenges -the affective gap, perception subjectivity, and label noise and absence. We begin with an introduction to the key emotion representation models that have been widely employed in AICA and description of available datasets for performing evaluation with quantitative comparison of label noise and dataset bias. We then summarize and compare the representative approaches on (1) emotion feature extraction, including both handcrafted and deep features, (2) learning methods on dominant emotion recognition, personalized emotion prediction, emotion distribution learning, and learning from noisy data or few labels, and (3) AICA based applications. Finally, we discuss some challenges and promising research directions in the future, such as image content and context understanding, group emotion clustering, and viewer-image interaction.

show abstract

Section: Learning-based Methodsmentioning

confidence: 99%

Affective Image Content Analysis: Two Decades Review and New Perspectives

Zhao

Yao

Yang

et al. 2022

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

show abstract

“…Woo et al [17] extend it and proposes a Convolutional block attention module that implements adaptively recalibrates spatial-wise feature responses. In recent years, inspired by channel and spatial attention, visual attention has been increasingly used in image sentiment analysis [18,19] . Unlike previous work, we develop a new attention module, Hybrid, which simultaneously models attribute dependencies between feature maps at adjacent semantic levels in the channel and spatial dimensions.…”

Section: Attention Mechanismmentioning

confidence: 99%

Exploring affective image representation with visual attention and aesthetic fusion

JiXiong,

Hao,

KangJian

et al. 2023

Fourteenth International Conference on Graphics and Image Processing (ICGIP 2022)

View full text Add to dashboard Cite

Affective image analysis aims to understand the sentiment of different images. The challenge is to develop a discriminative representation that bridges the affective gap between low-level features and high-level emotions. Most existing studies bridge the gap by designing deep models carefully to learn global representations in one shot directly or identify image emotion by extracting features at different levels in the model. They ignore that both local regions of an image and relationships between them impact emotional representation learning. This paper develops an affective image analysis method based on the aesthetic fusion hybrid attention network (AFHA). A modular hybrid attention block is designed to extract image emotion features and model long-range dependencies of images. By stacking hybrid attention blocks in ResNet-style, we obtain an affective representation backbone. Furthermore, considering that image emotion is inseparable from aesthetics, we employ a modified ResNet to extract image aesthetics. Finally, through a fusion strategy, the image's emotion is considered with the aesthetics conveyed. Experiments demonstrate the close relationship between emotion and aesthetics, and our plan has an excellent competitive effect compared with existing methods on the image sentiment analysis dataset.

show abstract

“…Self-attention is a crucial method for building relationships between queries and key features, which is beneficial for exploring the association between visual regions. He et al extracted local visual features by pyramid network and mined the association between local visual features through a self-attention mechanism [ 30 ]. Bera et al extracted semantic regions using SIFT key points and focused on the most relevant regions utilizing attention mechanisms [ 31 ].…”

Section: Related Workmentioning

confidence: 99%

Attention-Based Sentiment Region Importance and Relationship Analysis for Image Sentiment Recognition

Yang

Xing

Chang

et al. 2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Image sentiment recognition has attracted considerable attention from academia and industry due to the increasing tendency of expressing opinions via images and videos online. Previous studies focus on multilevel representation from global and local views to improve recognition performance. However, it is insufficient to research the importance and relationship of visual regions for image sentiment recognition. This paper proposes an attention-based sentiment region importance and relationship (ASRIR) analysis method, including important attention and relation attention for image sentiment recognition. First, we extract spatial region features using a multilevel pyramid network from the image. Second, we design important attention to exploring the sentiment semantic-related regions and relation attention to investigating the relationship between regions. In order to release the excessive concentration of attention, we employ a unimodal function as the objective function for regularization. Finally, the region features weighted by the attention mechanism are fused and input into a fully connected layer for classification. Extensive experiments on various commonly used image sentiment datasets demonstrate that our proposed method outperforms the state-of-the-art approaches.

show abstract

A Multi-Attentive Pyramidal Model for Visual Sentiment Analysis

Cited by 20 publications

References 26 publications

Affective Image Content Analysis: Two Decades Review and New Perspectives

Affective Image Content Analysis: Two Decades Review and New Perspectives

Exploring affective image representation with visual attention and aesthetic fusion

Attention-Based Sentiment Region Importance and Relationship Analysis for Image Sentiment Recognition

Contact Info

Product

Resources

About