Boosting image sentiment analysis with visual attention

Song, Kaikai; Yao, Ting; Ling, Qiang; Mei, Tao

doi:10.1016/j.neucom.2018.05.104

Cited by 136 publications

(60 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Schuch et al [39] used congruency sequence effect and a hanker interface paradigm and stroop-like interface for identifying conflict adaptation. Song et al [40] used convolutional networks for sentiment classification framework. It achieved good results when compared to the state-of-theart techniques.…”

Section: Literature Survey On Sarcasm Detectionmentioning

confidence: 99%

Multi-Rule Based Ensemble Feature Selection Model for Sarcasm Type Detection in Twitter

Sundararajan

Palanisamy

2020

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Sentimental analysis aims at inferring how people express their opinion over any piece of text or topic of interest. This article deals with detection of an implicit form of the sentiment, referred to as sarcasm. Sarcasm conveys the opposite of what people try to convey in order to criticize or ridicule in a humorous way. It plays a vital role in social networks since most of the tweets or posts contain sarcastic nuances. Existing approaches towards the study of sarcasm deals only with the detection of sarcasm. In this paper, in addition to detecting sarcasm from text, an approach has been proposed to identify the type of sarcasm. The main motivation behind determining the types of sarcasm is to identify the level of hurt or the true intent behind the sarcastic text. The proposed work aims to improve upon the existing approaches by incorporating a new perspective which classifies the sarcasm based on the level of harshness employed. The major application of the proposed work would be relating the emotional state of a person to the type of sarcasm exhibited by him/her which could provide major insights about the emotional behavior of a person. An ensemble-based feature selection method has been proposed for identifying the optimal set of features needed to detect sarcasm from tweets. This optimal set of features was employed to detect whether the tweet is sarcastic or not. After detecting sarcastic sentences, a multi-rule based approach has been proposed to determine the type of sarcasm. As an initial attempt, sarcasm has been classified into four types, namely, polite sarcasm, rude sarcasm, raging sarcasm, and deadpan sarcasm. The performance and efficiency of the proposed approach has been experimentally analyzed, and change in mood of a person for each sarcastic type has been modelled. The overall accuracy of the proposed ensemble feature selection algorithm for sarcasm detection is around 92.7%, and the proposed multi-rule approach for sarcastic type identification achieves an accuracy of 95.98%, 96.20%, 99.79%, and 86.61% for polite, rude, raging, and deadpan types of sarcasm, respectively.

show abstract

Section: Literature Survey On Sarcasm Detectionmentioning

confidence: 99%

Multi-Rule Based Ensemble Feature Selection Model for Sarcasm Type Detection in Twitter

Sundararajan

Palanisamy

2020

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

show abstract

“…Twitter I: The Twitter I dataset is the most popular image sentiment benchmark with 1,269 images collected from tweets [43]. Each image was labeled with a sentiment label (e.g., positive and negative) by five Amazon Mechanical Turk (AMT) works.…”

Section: A Datasetsmentioning

confidence: 99%

Visual Sentiment Analysis With Active Learning

2020

View full text Add to dashboard Cite

Visual Sentiment Analysis (VSA) has attracted wide attention since more and more people are willing to express their emotion and opinions via visual contents on social media. Meanwhile, extensive datasets drive the rapid development of deep neural networks for this task. However, the annotation of large-scale datasets is very expensive and time consuming. In this paper, we propose a novel active learning framework, which uses few labeled training samples to achieve an effective sentiment analysis model. First, we attach a new branch to the traditional Convolution Neural Network (CNN), which is named "texture module". The affective vector will be obtained by computing inner products of feature maps from different convolutional blocks in this branch. We will utilize this vector to distinguish affective images. Second, the query strategy is formed by the classification scores from both the traditional CNN and the texture module. Then, we can use samples obtained by utilizing the query strategy to train our model. Extensive experiments on four public affective datasets show that our approach uses few labeled training samples to achieve promising results for VSA.

show abstract

“…In the paper by Rao et al [61], a fast R-CNN located the distinctive parts of an image. The approach by Song et al [62] combined attention and salience. The procedure extracted a pair of maps for salience and attention from an input image and worked out a score based on the correspondence between the pair.…”

Section: Polarity Detectionmentioning

confidence: 99%

A Survey on Deep Learning in Image Polarity Detection: Balancing Generalization Performances and Computational Costs

et al. 2019

View full text Add to dashboard Cite

Deep convolutional neural networks (CNNs) provide an effective tool to extract complex information from images. In the area of image polarity detection, CNNs are customarily utilized in combination with transfer learning techniques to tackle a major problem: the unavailability of large sets of labeled data. Thus, polarity predictors in general exploit a pre-trained CNN as the feature extractor that in turn feeds a classification unit. While the latter unit is trained from scratch, the pre-trained CNN is subject to fine-tuning. As a result, the specific CNN architecture employed as the feature extractor strongly affects the overall performance of the model. This paper analyses state-of-the-art literature on image polarity detection and identifies the most reliable CNN architectures. Moreover, the paper provides an experimental protocol that should allow assessing the role played by the baseline architecture in the polarity detection task. Performance is evaluated in terms of both generalization abilities and computational complexity. The latter attribute becomes critical as polarity predictors, in the era of social networks, might need to be updated within hours or even minutes. In this regard, the paper gives practical hints on the advantages and disadvantages of the examined architectures both in terms of generalization and computational cost.

show abstract

Boosting image sentiment analysis with visual attention

Cited by 136 publications

References 13 publications

Multi-Rule Based Ensemble Feature Selection Model for Sarcasm Type Detection in Twitter

Multi-Rule Based Ensemble Feature Selection Model for Sarcasm Type Detection in Twitter

Visual Sentiment Analysis With Active Learning

A Survey on Deep Learning in Image Polarity Detection: Balancing Generalization Performances and Computational Costs

Contact Info

Product

Resources

About