Convolutional Neural Networks for Multimedia Sentiment Analysis

Cai, Guoyong; Xia, Baojia

doi:10.1007/978-3-319-25207-0_14

Cited by 95 publications

(55 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SentiBank & SentiStrength [2] extracts 1200 adjective-noun pairs as the middle-level features of image and calculates the sentiment scores based on English grammar and spelling style of texts. CNN-Multi [3] learns textual features and visual features by applying two individual CNN, and uses another CNN to exploiting the internal relation between text and image for sentiment classification. DNN-LR [27] trains a CNN for text and employs a deep convolutional neural network for image, and uses average strategy to aggregate probabilistic results which is the output of logistics regression.…”

Section: Compared Methodsmentioning

confidence: 99%

“…With the development of deep learning, deep neural networks have been employed for multimodal sentiment classification. Cai et al [3] and Yu et al [27] use CNN-based networks to extract feature representations from texts and images, and achieve significant progress. In order to model the relatedness between text and image, Xu et al [22] extract scene and object features from image, and absorb text words with these visual semantic features.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Fusion-Extraction Network for Multimodal Sentiment Analysis

Jiang

Wang

Liu

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Multiple modality data bring new challenges for sentiment analysis, as combining varieties of information in an effective manner is a rigorous task. Previous works do not effectively utilize the relationship and influence between texts and images. This paper proposes a fusion-extraction network model for multimodal sentiment analysis. First, our model uses an interactive information fusion mechanism to interactively learn the visual-specific textual representations and the textual-specific visual representations. Then, we propose an information extraction mechanism to extract valid information and filter redundant parts for the specific textual and visual representations. The experimental results on two public multimodal sentiment datasets show that our model outperforms existing state-of-the-art methods.

show abstract

Section: Compared Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Fusion-Extraction Network for Multimodal Sentiment Analysis

Jiang

Wang

Liu

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…The authors in paper [5] have proposed another CNN architecture that completely utilizes joint text-level and imagelevel portrayal to perform mixed media sentiment analysis. In light of thought of the correlative impact of the two portrayals as sentiment features, the proposed strategy takes the benefit of the inner connection amongst text and image in image tweets and uses it to accomplish better performance results in sentiment prediction.…”

Section: Related Workmentioning

confidence: 99%

Detection of Visual Positive Sentiment using PCNN

Ahmed¹,

Nabil²,

Badr³

2019

ijacsa

View full text Add to dashboard Cite

Many people all over the world use online social networks to express their feeling and sharing their experience, and the easiest way from their perspective is using images and videos to do so. This paper shows the utilization of two techniques (Viola et al algorithm and Pulse coupled Neural Network) in visual sentiment analysis using a hand-labeled dataset. The proposed system, which uses the PCNN with NN classifier, achieves 96% right classification, whereas Viola algorithm achieves 94% for the same dataset.

show abstract

“…Mandel et al () trained several classifiers on the messages collected from Hurricane Irene in Twitter to recognize the sentiment of the public. However, in multimedia era, online users are more willing to post blogs attached with images to express their feelings on social platforms (Cai & Xia, ). Furthermore, it is insufficient to recognize online users' emotions just through analyzing the texts.…”

Section: Introductionmentioning

confidence: 99%

The sentiment recognition of online users based on DNNs multimodal fusion

Fan

Zhu

2019

Proceedings of the Association for Information Science and Tech

View full text Add to dashboard Cite

Online users are more willing to express their opinions and emotions on social platforms in online public opinion events, through combing texts and images rather than texts alone. Facing the change of emotional expressions, it is extremely vital to recognize the sentiments of online users in online public opinion events in an appropriate way. In this paper, we propose a novel Deep Neural Networks (DNNs) model to recognize online users' sentiments in online public opinion events by analyzing the sentiments of texts and attached images together. Also, we compare two fusion strategies, feature‐level fusion and decision‐level fusion, to combine the affective information from texts and images. In feature‐level fusion, fine‐tuned Convolutional Neural Networks (CNNs) and Bidirectional Long Short‐Term Memory (BiLSTMs) are employed to extract visual and textual features respectively. Then the features are concatenated and fed to a DNNs system to complete classification. In decision‐level fusion, a rule is employed to fuse the output from unimodality, generating final predicted labels. Experimental results showed that the proposed DNNs multimodal model achieved better performance than unimodal sentiment recognition model. In fusion strategy, the feature‐level fusion performed better in our experiments.

show abstract

Convolutional Neural Networks for Multimedia Sentiment Analysis

Cited by 95 publications

References 16 publications

Fusion-Extraction Network for Multimodal Sentiment Analysis

Fusion-Extraction Network for Multimodal Sentiment Analysis

Detection of Visual Positive Sentiment using PCNN

The sentiment recognition of online users based on DNNs multimodal fusion

Contact Info

Product

Resources

About