Detecting fake news stories via multimodal analysis

Abstract

Filtering, vetting, and verifying digital information is an area of core interest in information science. Online fake news is a specific type of digital misinformation that poses serious threats to democratic institutions, misguides the public, and can lead to radicalization and violence. Hence, fake news detection is an important problem for information science research. While there have been multiple attempts to identify fake news, most of such efforts have focused on a single modality (e.g., only text‐based or only visual features). However, news articles are increasingly framed as multimodal news stories, and hence, in this work, we propose a multimodal approach combining text and visual analysis of online news stories to automatically detect fake news. Drawing on key theories of information processing and presentation, we identify multiple text and visual features that are associated with fake or credible news articles. We then perform a predictive analysis to detect features most strongly associated with fake news. Next, we combine these features in predictive models using multiple machine‐learning techniques. The experimental results indicate that a multimodal approach outperforms single‐modality approaches, allowing for better fake news detection.

Predicting image credibility in fake news over social media using multi-modal approach

“…Recently, research has been done using multi modalities which perform better than single modalities [ 9 , 10 ]. Jin et al [ 37 ] integrated multiple content types and suggested solution using a recurrent neural network (RNN) having an attention mechanism for combining features of the visual, textual, and social context.…”

“…A multi-modal approach has recently been used, which uses multiple content and context type like text, visual, statistical, user profile, and network propagation to detect fake news. Out of these, a multi-modal framework using image and text has fared a little better than others [ 9 , 10 ].…”

ReOpen demands as public health threat: a sociotechnical framework for understanding the stickiness of misinformation

“…The ability to access and analyze large datasets has transformed social scientific research. Data science helps researchers understand actions, behaviors, and choices within networked publics over short periods of time and provides the opportunity to understand misinformation ecosystems, detect bot-driven activities, and track social movements and activism within and across platforms (Benkler et al 2018 ; Freelon et al 2016 ; Freelon et al 2020 ; Jackson et al 2020 ; Lazer et al 2020 ; Singh et al 2020 ; Tacchini et al 2017 ; Uyheng and Carley 2020 ).…”

Forensic Analysis of Memetic Image Propagation: Introducing the SMOC BRISQUEt Method

“…Computer and information science has tackled these issues by automating various forms of content identification, evaluation, and classification, with a particular focus on identifying content that is false or misleading (Farid 2006, Gupta et al 2013, Ma et al 2013, Conroy et al 2015, Rubin 2016. A more recently emerging and still-developing third thread of research also exists, which combines the aforementioned humanistic and computational approaches to understanding memetic distribution (Teyssou et al 2017, Acker and Chaiet 2020, Singh et al 2020.…”