Emotional Attention: A Study of Image Sentiment and Visual Attention

Fan, Shaosheng; Shen, Zhiqi; Jiang, Ming; Koenig, Bryan L.; Xu, Jing; Kankanhalli, Mohan; Zhao, Qi

doi:10.1109/cvpr.2018.00785

Cited by 131 publications

(91 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…IAPSa [12] 395 8 N Abstract [12] 228 8 N ArtPhoto [12] 806 8 N Twitter I [19] 1,269 2 N Twitter II [31] 603 2 N EmotionROI [14] 1,980 6 Y EMOd [15] 1,019 10 Y Flickr&Instagram [10] 23,308 8 N Flickr [43] 60,745 2 N Instagram [43] 42,856 2 N mapping sentiments into intuitive categories [26]- [28]. In the early years, there are numerous methods using hand-crafted features for image sentiment classification [11], [29], [30].…”

Section: Dataset #Images #Classes Regionsmentioning

confidence: 99%

“…Fig. 1 shows examples from the EmotionROI [14] and EMOd datasets [15]. As can be seen, specific regions show strong influence on evoked sentiment.…”

Section: Introductionmentioning

confidence: 98%

“…A preliminary version of this work appeared in CVPR [1]. Examples from the (a) EmotionROI [14] and (b) EMOd datasets [15] with the human annotation. The ground-truth sentiments are also given.…”

Section: Introductionmentioning

confidence: 99%

“…First, compared with conventional recognition tasks, visual sentiment analysis is more challenging due to a higher level of subjectivity in the human recognition process [2]. Neuroimaging and behavioral studies find that human attention is attracted by emotional relevance of a stimulus [16]- [18], which is also proved as the emotion prioritization effect in computer vision studies [15]. Fig.…”

Section: Introductionmentioning

confidence: 99%

“…(2) We add a comprehensive review of related work making the manuscript more self-contained. (3) We conduct an exhaustive analysis on the weakly supervised detection framework for visual sentiment prediction and add evaluation on the eye-tracking dataset [15]. For comparison, recent learning based salience detection methods are also trained and evaluated on such datasets.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

WSCNet: Weakly Supervised Coupled Networks for Visual Sentiment Classification and Detection

She

Yang

Cheng

et al. 2020

IEEE Trans. Multimedia

View full text Add to dashboard Cite

Automatic assessment of sentiment from visual content has gained considerable attention with the increasing tendency of expressing opinions online. In this paper, we solve the problem of visual sentiment analysis, which is challenging due to the high-level abstraction in the recognition process. Existing methods based on convolutional neural networks learn sentiment representations from the holistic image, despite the fact that different image regions can have different influence on the evoked sentiment. In this paper, we introduce a weakly supervised coupled convolutional network (WSCNet). Our method is dedicated to automatically selecting relevant soft proposals given weak annotations (e.g., global image labels), thereby significantly reducing the annotation burden, and encompasses the following contributions. First, the proposed WSCNet detects a sentimentspecific soft map by training a fully convolutional network with the cross spatial pooling strategy in the detection branch. Second, both the holistic and localized information are utilized by coupling the sentiment map with deep features as semantic vector in the classification branch. The sentiment detection and classification branches are integrated into a unified deep framework optimized in an end-to-end manner. Extensive experiments demonstrate that the proposed WSCNet outperforms the stateof-the-art results on seven benchmark datasets.

show abstract

Section: Dataset #Images #Classes Regionsmentioning

confidence: 99%

“…Fig. 1 shows examples from the EmotionROI [14] and EMOd datasets [15]. As can be seen, specific regions show strong influence on evoked sentiment.…”

Section: Introductionmentioning

confidence: 98%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

WSCNet: Weakly Supervised Coupled Networks for Visual Sentiment Classification and Detection

She

Yang

Cheng

et al. 2020

IEEE Trans. Multimedia

View full text Add to dashboard Cite

show abstract

Various syncretic co‐attention network for multimodal sentiment analysis

Cao

Zhu

Gao

et al. 2020

Concurrency and Computation

View full text Add to dashboard Cite

The multimedia contents shared on social network reveal public sentimental attitudes toward specific events. Therefore, it is necessary to conduct sentiment analysis automatically on abundant multimedia data posted by the public for real-world applications. However, approaches to single-modal sentiment analysis neglect the internal connections between textual and visual contents, and current multimodal methods fail to exploit the multilevel semantic relations of heterogeneous features. In this article, the various syncretic co-attention network is proposed to excavate the intricate multilevel corresponding relations between multimodal data, and combine the unique information of each modality for integrated complementary sentiment classification. Specifically, a multilevel co-attention module is constructed to explore localized correspondences between each image region and each text word, and holistic correspondences between global visual information and context-based textual semantics. Then, all the single-modal features can be fused from different levels, respectively. Except for fused multimodal features, our proposed VSCN also considers unique information of each modality simultaneously and integrates them into an end-to-end framework for sentiment analysis. The superior results of experiments on three constructed real-world datasets and a benchmark dataset of Visual Sentiment Ontology (VSO) prove the effectiveness of our proposed VSCN. Especially qualitative analyses are given for deep explaining of our method.

show abstract

Weakly Supervised Learning of Image Emotion Analysis Based on Cross-spatial Pooling

Peng

2019

Biometric Recognition

View full text Add to dashboard Cite

Emotional Attention: A Study of Image Sentiment and Visual Attention

Cited by 131 publications

References 48 publications

WSCNet: Weakly Supervised Coupled Networks for Visual Sentiment Classification and Detection

WSCNet: Weakly Supervised Coupled Networks for Visual Sentiment Classification and Detection

Various syncretic co‐attention network for multimodal sentiment analysis

Weakly Supervised Learning of Image Emotion Analysis Based on Cross-spatial Pooling

Contact Info

Product

Resources

About