Multi-Modal Sentiment Classification With Independent and Interactive Knowledge via Semi-Supervised Learning

Zhang, Dong; Li, Shoushan; Zhu, Qiaoming; Zhou, Guodong

doi:10.1109/access.2020.2969205

Cited by 22 publications

(12 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yu et al [2,3] first introduced the multimodal factorized bilinear pooling approach to integrating image and text features and then used the co-attention mechanism to jointly learn the image and text attentions. Moreover, Zhang et al [27] developed semi-supervised variational autoencoders to extract independent knowledge from single-modality data and interactive knowledge from different modalities. Furthermore, Wang et al [28] proposed an end-to-end fusion method to perform the MSC task.…”

Section: Related Workmentioning

confidence: 99%

Cross-modal complementary network with hierarchical fusion for multimodal sentiment classification

Peng

Zhang

Xue

et al. 2022

Tsinghua Sci. Technol.

View full text Add to dashboard Cite

Multimodal Sentiment Classification (MSC) uses multimodal data, such as images and texts, to identify the users' sentiment polarities from the information posted by users on the Internet. MSC has attracted considerable attention because of its wide applications in social computing and opinion mining. However, improper correlation strategies can cause erroneous fusion as the texts and the images that are unrelated to each other may integrate.Moreover, simply concatenating them modal by modal, even with true correlation, cannot fully capture the features within and between modals. To solve these problems, this paper proposes a Cross-Modal Complementary Network (CMCN) with hierarchical fusion for MSC. The CMCN is designed as a hierarchical structure with three key modules, namely, the feature extraction module to extract features from texts and images, the feature attention module to learn both text and image attention features generated by an image-text correlation generator, and the cross-modal hierarchical fusion module to fuse features within and between modals. Such a CMCN provides a hierarchical fusion framework that can fully integrate different modal features and helps reduce the risk of integrating unrelated modal features. Extensive experimental results on three public datasets show that the proposed approach significantly outperforms the state-of-the-art methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Cross-modal complementary network with hierarchical fusion for multimodal sentiment classification

Peng

Zhang

Xue

et al. 2022

Tsinghua Sci. Technol.

View full text Add to dashboard Cite

show abstract

“…Ji et al [29] designed a LSTM based semisupervised attention framework, experiments demonstrated that the possibility of the framework for sentiment analysis. Zhang et al [30] used semi-supervised learning to reduce the annotation work for multimodal sentiment classification, the effectiveness of the proposed semi-supervised approach was indicated in experiments.…”

Section: Related Workmentioning

confidence: 99%

Sentiment Analysis via Deep Multichannel Neural Networks With Variational Information Bottleneck

Luo

2020

IEEE Access

View full text Add to dashboard Cite

With the rapid development of e-commerce, online consumption has become a mainstream form of consumption in recent years. Text sentiment analysis for a large number of customer reviews on the e-commerce platform can dramatically improve the customers' consumption experience. Although the sentiment analysis approaches based on deep neural network can achieve higher accuracy without humandesign features compared with traditional sentiment analysis methods, the accuracy still cannot meet the demand and the training suffers from the issues of over-fitting, vanishing gradient, etc. In this paper, a novel sentiment analysis model named MBGCV is designed to alleviate these problems and improve the accuracy, MBGCV employs a multichannel paradigm and integrates Bidirectional Gated Recurrent Unit (BiGRU), Convolutional Neural Network (CNN) and Variational Information Bottleneck (VIB). The multichannel is exploited to extract multi-grained sentiment features. In each channel, BiGRU is utilized to extract context information, and then CNN is adopted to extract local features. Furthermore, the model combines the advantages of VIB and Maxout activation function to overcome shortcomings such as overfitting, vanishing gradient in existing sentiment analysis models. By using real review datasets, we carry out extensive experiments to demonstrate that our proposed model can achieve superior accuracy and improve the performance of text sentiment analysis.

show abstract

“…Zhang, etc. [28] aimed to reduce the annotation effort for multi-modal sentiment classification via semi-supervised learning method. Its key idea was to use the semi-supervised variational autoencoders to detect more information from unlabeled data for multi-modal sentiment analysis.…”

Section: Related Work a Sentiment Analysis And Its Classificationmentioning

confidence: 99%

A Multiclassification Model of Sentiment for E-Commerce Reviews

Zhang

Zhong

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Consumer reviews are important information that reflects the quality of E-commerce goods and services and their existing problems after shopping. Due to the possible differences in consumers' experiences with goods and service quality, consumer reviews can involve multiple-aspect expressions of emotions or opinions. This may result in attitudes expressed by a consumer in the same review sometimes having a variety of emotions. We introduce a sentiment multiclassification method based on a directed weighted model. The model represents the sentiment entity vocabulary as the sentiment nodes and represents the relation between nodes as the directed weighted link. The sentiment entity vocabulary is the entity with attributes, which can express sentiment meaning in related reviews. Directed weighted links represent the sentiment similarity between two nodes of entities with attributes and determined by the direct correlation calculation between them. The paths are all connected directed links from one node to another, which are composed of several nodes and links with close sentiment similarity. Then, we can establish a directed weighted model concerning the sentiments. Directed weighted links having similar sentiment relations with each other may constitute a directed weighted path. There are several directed weighted paths from a start node to the end nodes of the sentiment entity vocabulary in the directed weighted model. Each different path is a different sentiment expression, which represents a different sentiment type. The different sentiment classifications can be obtained through the restriction of path length. Experiments and analysis of the results show that the sentiment multiclassification model based on the directed weighted model proposed in this paper can classify the review sentiments according to different limited threshold rules. Comprehensive analysis indicates the classification results have good accuracy and high efficiency.

show abstract

Multi-Modal Sentiment Classification With Independent and Interactive Knowledge via Semi-Supervised Learning

Cited by 22 publications

References 26 publications

Cross-modal complementary network with hierarchical fusion for multimodal sentiment classification

Cross-modal complementary network with hierarchical fusion for multimodal sentiment classification

Sentiment Analysis via Deep Multichannel Neural Networks With Variational Information Bottleneck

A Multiclassification Model of Sentiment for E-Commerce Reviews

Contact Info

Product

Resources

About