Image sentiment analysis using latent correlations among visual, textual, and sentiment views

Katsurai, Marie; Satoh, Shin’ichi

doi:10.1109/icassp.2016.7472195

Cited by 81 publications

(48 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…IAPSa [12] 395 8 N Abstract [12] 228 8 N ArtPhoto [12] 806 8 N Twitter I [19] 1,269 2 N Twitter II [31] 603 2 N EmotionROI [14] 1,980 6 Y EMOd [15] 1,019 10 Y Flickr&Instagram [10] 23,308 8 N Flickr [43] 60,745 2 N Instagram [43] 42,856 2 N mapping sentiments into intuitive categories [26]- [28]. In the early years, there are numerous methods using hand-crafted features for image sentiment classification [11], [29], [30].…”

Section: Dataset #Images #Classes Regionsmentioning

confidence: 99%

“…We evaluate the proposed WSCNet on seven public affective datasets including the Flickr and Instagram (FI) [10], Flickr [43], Instagram [43], Twitter I [19], Twitter II [31], EmotionROI [14], EMOd datasets [15].…”

Section: A Datasetsmentioning

confidence: 99%

“…The FI datasets are split randomly into 80% training, 5% validation and 15% testing sets. For the Flickr dataset and Instagram dataset, we randomly sample the same number of images for each class following the same configuration in [43], which are split randomly into 90% training, 10% testing sets. The small-scale datasets are split into 80% training and 20% testing sets randomly except those with specified training/testing splits [14], [31].…”

Section: A Datasetsmentioning

confidence: 99%

See 2 more Smart Citations

WSCNet: Weakly Supervised Coupled Networks for Visual Sentiment Classification and Detection

She

Yang

Cheng

et al. 2020

IEEE Trans. Multimedia

View full text Add to dashboard Cite

Automatic assessment of sentiment from visual content has gained considerable attention with the increasing tendency of expressing opinions online. In this paper, we solve the problem of visual sentiment analysis, which is challenging due to the high-level abstraction in the recognition process. Existing methods based on convolutional neural networks learn sentiment representations from the holistic image, despite the fact that different image regions can have different influence on the evoked sentiment. In this paper, we introduce a weakly supervised coupled convolutional network (WSCNet). Our method is dedicated to automatically selecting relevant soft proposals given weak annotations (e.g., global image labels), thereby significantly reducing the annotation burden, and encompasses the following contributions. First, the proposed WSCNet detects a sentimentspecific soft map by training a fully convolutional network with the cross spatial pooling strategy in the detection branch. Second, both the holistic and localized information are utilized by coupling the sentiment map with deep features as semantic vector in the classification branch. The sentiment detection and classification branches are integrated into a unified deep framework optimized in an end-to-end manner. Extensive experiments demonstrate that the proposed WSCNet outperforms the stateof-the-art results on seven benchmark datasets.

show abstract

Section: Dataset #Images #Classes Regionsmentioning

confidence: 99%

Section: A Datasetsmentioning

confidence: 99%

Section: A Datasetsmentioning

confidence: 99%

See 1 more Smart Citation

WSCNet: Weakly Supervised Coupled Networks for Visual Sentiment Classification and Detection

She

Yang

Cheng

et al. 2020

IEEE Trans. Multimedia

View full text Add to dashboard Cite

show abstract

“…Image polarity detection can also apply to multimodal sentiment analysis. Remarkable results have been achieved in [74][75][76][77][78][79], where ensembles of handcrafted features were worked out from images and combined with information provided by text analysis. Those approaches were subsequently outperformed by frameworks that integrated CNNs for extracting features from visual content [12,78,[80][81][82][83][84][85].…”

Section: Sentiment Analysis: Other Applicationsmentioning

confidence: 99%

A Survey on Deep Learning in Image Polarity Detection: Balancing Generalization Performances and Computational Costs

et al. 2019

View full text Add to dashboard Cite

Deep convolutional neural networks (CNNs) provide an effective tool to extract complex information from images. In the area of image polarity detection, CNNs are customarily utilized in combination with transfer learning techniques to tackle a major problem: the unavailability of large sets of labeled data. Thus, polarity predictors in general exploit a pre-trained CNN as the feature extractor that in turn feeds a classification unit. While the latter unit is trained from scratch, the pre-trained CNN is subject to fine-tuning. As a result, the specific CNN architecture employed as the feature extractor strongly affects the overall performance of the model. This paper analyses state-of-the-art literature on image polarity detection and identifies the most reliable CNN architectures. Moreover, the paper provides an experimental protocol that should allow assessing the role played by the baseline architecture in the polarity detection task. Performance is evaluated in terms of both generalization abilities and computational complexity. The latter attribute becomes critical as polarity predictors, in the era of social networks, might need to be updated within hours or even minutes. In this regard, the paper gives practical hints on the advantages and disadvantages of the examined architectures both in terms of generalization and computational cost.

show abstract

“…Successful results in multimodal sentiment analysis have been achieved using non-negative matrix factorisation [43] and latent correlations [15]. Chen et al [5] investigated the image posting behaviour of social media users and found in their study that two thirds of the participants added an image to their tweets to enhance the emotion of the text.…”

Section: Related Workmentioning

confidence: 99%

Multimodal Sentiment Analysis To Explore the Structure of Emotions

Flaxman

2018

Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery &Amp; Data Mining

100

View full text Add to dashboard Cite

We propose a novel approach to multimodal sentiment analysis using deep neural networks combining visual analysis and natural language processing. Our goal is different than the standard sentiment analysis goal of predicting whether a sentence expresses positive or negative sentiment; instead, we aim to infer the latent emotional state of the user. Thus, we focus on predicting the emotion word tags attached by users to their Tumblr posts, treating these as "self-reported emotions. " We demonstrate that our multimodal model combining both text and image features outperforms separate models based solely on either images or text. Our model's results are interpretable, automatically yielding sensible word lists associated with emotions. We explore the structure of emotions implied by our model and compare it to what has been posited in the psychology literature, and validate our model on a set of images that have been used in psychology studies. Finally, our work also provides a useful tool for the growing academic study of images-both photographs and memes-on social networks.

show abstract

Image sentiment analysis using latent correlations among visual, textual, and sentiment views

Cited by 81 publications

References 22 publications

WSCNet: Weakly Supervised Coupled Networks for Visual Sentiment Classification and Detection

WSCNet: Weakly Supervised Coupled Networks for Visual Sentiment Classification and Detection

A Survey on Deep Learning in Image Polarity Detection: Balancing Generalization Performances and Computational Costs

Multimodal Sentiment Analysis To Explore the Structure of Emotions

Contact Info

Product

Resources

About