Visual Sentiment Analysis With Active Learning

Abstract

Visual Sentiment Analysis (VSA) has attracted wide attention since more and more people are willing to express their emotion and opinions via visual contents on social media. Meanwhile, extensive datasets drive the rapid development of deep neural networks for this task. However, the annotation of large-scale datasets is very expensive and time consuming. In this paper, we propose a novel active learning framework, which uses few labeled training samples to achieve an effective sentiment analysis model. First, we attach a new branch to the traditional Convolution Neural Network (CNN), which is named "texture module". The affective vector will be obtained by computing inner products of feature maps from different convolutional blocks in this branch. We will utilize this vector to distinguish affective images. Second, the query strategy is formed by the classification scores from both the traditional CNN and the texture module. Then, we can use samples obtained by utilizing the query strategy to train our model. Extensive experiments on four public affective datasets show that our approach uses few labeled training samples to achieve promising results for VSA.

Image and Text Aspect Level Multimodal Sentiment Classification Model Using Transformer and Multilayer Attention Interaction

“…They used the Dense Net-169 model based on deep CNN, and they adopted chimpanzee optimization algorithm as the super parameter tuning method, using teaching and learning based optimization combined with LSTM model for facial expression recognition and classification. Chen et al (2020) added a new branch named "texture module" to the traditional CNN. By using this branch and different feature maps to calculate sentiment vectors, they achieved image sentiment differentiation.…”

Multi-Model Fusion Framework Using Deep Learning for Visual-Textual Sentiment Classification

“…SA is a specialized form of Natural Language Processing (NLP) that can be applied in a wide range of real-world applications, including financial and stock price predictions [1,2], politics [3], medicine [4], and e-tourism [5]. Many researchers have devoted substantial efforts to studying textual SA [6][7][8][9][10] using different techniques, along with visual communication, which has remarkably developed on social media platforms [11][12][13][14][15][16]. However, most described studies have only assessed information from one modality and ignored the rich and complimentary sentiment information in multimodal data.…”

Develop an Automatic System based on Object Recognition Techniques for Predicting Visual Sentiments from Social Network Images

“…It is one of the computer vision tasks. It aims to analyze images and enable computers to understand what is happening in those images to detect and express the positive and negative sentiments and opinions conveyed [4]. It entails being able to recognize the objects and scenes in images, as well as their emotional context from images.…”