Predicting image credibility in fake news over social media using multi-modal approach

Abstract

Social media are the main contributors to spreading fake images. Fake images are manipulated images altered through software or by other means to change the information they convey. Fake images propagated over microblogging platforms generate misrepresentation and stimulate polarization in the people. Detection of fake images shared over social platforms is extremely critical to mitigating its spread. Fake images are often associated with textual data. Hence, a multi-modal framework is employed utilizing visual and textual feature learning. However, few multi-modal frameworks are already proposed; they are further dependent on additional tasks to learn the correlation between modalities. In this paper, an efficient multi-modal approach is proposed, which detects fake images of microblogging platforms. No further additional subcomponents are required. The proposed framework utilizes explicit convolution neural network model EfficientNetB0 for images and sentence transformer for text analysis. The feature embedding from visual and text is passed through dense layers and later fused to predict fake images. To validate the effectiveness, the proposed model is tested upon a publicly available microblogging dataset, MediaEval (Twitter) and Weibo, where the accuracy prediction of 85.3% and 81.2% is observed, respectively. The model is also verified against the newly created latest Twitter dataset containing images based on India's significant events in 2020. The experimental results illustrate that the proposed model performs better than other state-of-art multi-modal frameworks.

FakedBits- Detecting Fake Information on Social Platforms using Multi-Modal Features

“…Therefore, we have used both distilBERT and EfficientNetB0 to achieve high accuracy. Authors [14] utilized distilBERT instead of RoBERTa for a late fusion strategy for a multi-modal fusion.…”

A systematic literature review and existing challenges toward fake news detection models

“…In the digital era, it is simpler for spreading fake news while a user may distribute fake news to neighbors, their friends, and so on due to the unique characteristics of social media (Habib et al 2019). Thus, fake news can be propagated in a cycle format because of the vast usage of social media by every individual (Singh and Sharma 2021). Moreover, comments on fake news can be varied every time that reducing the reliability of real news, the fake news has directly spread in a faster way while comparing to real news (Yang et al 2021).…”

Detecting and locating trending places using multimodal social network data

“…There are also important contributions In the field of sentiment analysis [4,15,16], merging audio, video and image inputs, and also using Transformer models in the case of [15]. More natural language processing applications include the generation of dialogues combining video and text inputs [19], summary generation from audio and video [41], video retrieval [10], and fake news detection [31].…”