Multidimensional Extra Evidence Mining for Image Sentiment Analysis

Abstract

Image sentiment analysis is a hot research topic in the field of computer vision. However, two key issues need to be addressed. First, high-quality training samples are scarce. There are numerous ambiguous images in the original datasets owing to diverse subjective cognitions from different annotators. Second, the cross-modal sentimental semantics among heterogeneous image features has not been fully explored. To alleviate these problems, we propose a novel model called multidimensional extra evidence mining (ME 2 M) for image sentiment analysis, it involves sample-refinement and cross-modal sentimental semantics mining. A new soft voting-based sample-refinement strategy is designed to address the former problem, whereas the state-of-the-art discriminant correlation analysis (DCA) model is used to completely mine the cross-modal sentimental semantics among diverse image features. Image sentiment analysis is conducted based on the cross-modal sentimental semantics and a general classifier. The experimental results verify that the ME 2 M model is effective and robust and that it outperforms the most competitive baselines on two well-known datasets. Furthermore, it is versatile owing to its flexible structure.

DCML: Deep contrastive mutual learning for COVID-19 recognition

“…Hence, robust but effective feature learning (or feature fusion strategy) is more important for this dataset. Another sample refinement method [67] may address this issue well. Fourth, using the contrastive loss only brings evident performance improvements on the COVID-19_Radiography_Dataset datasets.…”

Image Sentiment Analysis via Active Sample Refinement and Cluster Correlation Mining

“…Moreover, any single image feature is hard to integrally describe an image. Hence, Zhang et al [13] proposed a feature mid-fusion algorithm called gene selection XGBoost (GS-XGB) to mine the implicit correlation among a group of image features [14], which obtained satisfactory performance of image sentiment analysis. However, the feature mid-fusion algorithm requires numerous features and is intricate to realize.…”

“…Furthermore, it separates the inherent dependencies between different classes. Unlike feature mid-fusion, Zhang et al [14] employed discriminant correlation analysis (DCA) [15] to complete feature early-fusion and mine the correlation among a set of heterogeneous features.…”

“…e corresponding ME 2 M model [14] is effective and efficient. But the feature dimension after the DCA operation is too low, which loses some important information.…”

“…e above studies [10][11][12][13][14] are fully supervised, which need massive accurate sentiment annotations.…”