MM-Claims: A Dataset for Multimodal Claim Detection in Social Media

Abstract

In recent years, the problem of misinformation on the web has become widespread across languages, countries, and various social media platforms. Although there has been much work on automated fake news detection, the role of images and their variety are not well explored. In this paper, we investigate the roles of image and text at an earlier stage of the fake news detection pipeline, called claim detection. For this purpose, we introduce a novel dataset, MM-Claims, which consists of tweets and corresponding images over three topics: COVID-19, Climate Change and broadly Technology. The dataset contains roughly 86 000 tweets, out of which 3400 are labeled manually by multiple annotators for the training and evaluation of multimodal models. We describe the dataset in detail, evaluate strong unimodal and multimodal baselines, and analyze the potential and drawbacks of current models.

Robust and Efficient Claim Retrieval for Online Fact-Checking Applications

“…Check square [45] builds a KD-tree based on embeddings for claim texts and titles to retrieve the 1,000 most similar verified claims for an input claim. We report results for two of their variants which performed better than their primary submission: one using Sentence-BERT-Large pre-trained on SNLI with MAX tokens, fine-tuned with triplet loss, and the other one using multilingual DistilBERT-embeddings without fine-tuning (distmult).…”

“…Our results generally verify the observations made in the literature that the choice of language models for this task greatly impacts the results: For Check square, the authors found on the 2021-tweets data that multilingual DistilBERT embeddings without fine-tuning outperformed fine-tuned monolingual Sentence-BERT models. For the former, fine-tuning using their triplet loss methodology hurts performance while it helps in the case of Sentence-BERT [45]. Similarly, [39] found on the same dataset that their RoBERTa model fine-tuned on triplets performed worse than monolingual DistilBERT without fine-tuning.…”

Multimodal Automated Fact-Checking: A Survey

“…The term fauxtography was first coined in journalism for images manipulated to "convey a questionable (or outright false) sense of the events they seem to depict" (Cooper, 2007;Kalb and Saivetz, 2007). Other terms used in the literature for manipulated content are fake (Cheema et al, 2022), forgery (Cozzolino et al, 2021), and splice (Zampoglou et al, 2015).…”

“…Claim detection is typically framed as a classification task. Models predict if a claim is checkable or check-worthy (Prabhakar et al, 2021;Cheema et al, 2022;Barrón-Cedeño et al, 2023). The verdict for factual-verifiability is often binary (Jin et al, 2017;Shang et al, 2021).…”