FauxWard: a graph neural network approach to fauxtography detection using social media comments

Abstract

Online social media has been a popular source for people to consume and share news content. More recently, the spread of misinformation online has caused widespread concerns. In this work, we focus on a critical task of detecting fauxtography on social media where the image and associated text together convey misleading information. Many efforts have been made to mitigate misinformation online, but we found that the fauxtography problem has not been fully addressed by existing work. Solutions focusing on detecting fake images or misinformed texts alone on social media often fail to identify the misinformation delivered together by the image and the associated text of a fauxtography post. In this paper, we develop FauxWard, a novel graph convolutional neural network framework that explicitly explores the complex information extracted from a user comment network of a social media post to effectively identify fauxtography. FauxWard is content-free in the sense that it does not analyze the visual or textual contents of the post itself, which makes it robust against sophisticated fauxtography uploaders who intentionally craft image-centric posts by editing either the text or image content. We evaluate FauxWard on two real-world datasets collected from mainstream social media platforms (i.e., Reddit and Twitter). The results show that FauxWard is both effective and efficient in identifying fauxtography posts on social media.

What and Why? Towards Duo Explainable Fauxtography Detection Under Constrained Supervision

“…Zhang et al [10] proposed FxBuster, a content-free fauxtography detector that explores the user comment network and their emotions to identify fauxtography posts. Shang [11] extended the FxBuser by leveraging the heterogeneous information (e.g., the linguistic and semantic comment information) in comments of posts to further improve the fauxtography detection performance. Zlatkova et al [7] developed a content based approach that identifies the fauxtography posts by exploring the URLs of images in the posts.…”

“…In particular, the comments are modeled as graph nodes and the "reply" relations are modeled as graph edges in the network. However, in many cases, only considering direct "reply" between user comments (e.g., ExFaux [18], FauxWard [11]) is insufficient because such direct "reply" is often either sparse (e.g., few discussions under the post) or long-chain (e.g., a long debate between two users) in reality [41]. One possible solution is to connect each comment with all other comments in the same thread of the given post as indirect "reply".…”

“…Recently, several initial efforts have been made to study the problem of fauxtography detection in big data analytics and machine learning [6], [7], [9], [10], [11]. However, those solutions only focus on using the social media posts with binary fauxtography labels (i.e., fauxtography or nonfauxtography) as the training data and could only estimate if a post is fauxtography or not without explaining their decisions (i.e., what specific component of the post is false and why that component is false).…”

COSMOS: Catching Out-of-Context Misinformation with Self-Supervised Learning

“…However, these methods target only textual claims and therefore cannot be directly applied to claims about images. To detect the increas-ing number of false claims about images, a few methods [19,50,39,46,53,20] have been proposed recently. For instance, Khattar et al [20] learn a variational autoencoder based on shared embedding space (textual and visual) with binary classifier to detect fake news.…”

MMSys'21 Grand Challenge on Detecting Cheapfakes

“…Note that OOC misuse of images should not be confused with multi-modal (image, caption) fake news detection methods [17,18,28,33,37,38], which aim to identify fake news where images could be photoshopped and the real counterpart does not even exists. In the case of OOC misuse, the images are genuine and the real counterpart always exists.…”