Hateful meme detection with multimodal deep neural networks

Abstract

The modern advances of social media platforms and content sharing websites led to the popularization of Internet memes, and today's Internet landscape contains websites that are predominantly dedicated to meme sharing. While at their inception memes were mostly humorous, this concept evolved and nowadays memes cover a wide variety of subjects, including political and social commentaries. Considering the widespread use of memes and their power of conveying distilled messages, they became an important method for spreading hate speech against individuals or targeted groups. Given the multimodal nature of Internet memes, our proposed approach is also a multimodal one, consisting of two parallel processing branches, one textual and one visual, that are joined in a final classification step, providing prediction results for the samples. We test our approach on the publicly available Memotion 7k dataset and compare our results with the baseline approach developed for the dataset.

Multimodal Hate Speech Detection in Memes Using Contrastive Language-Image Pre-Training

“…Table 3 shows the results of a comparison between the evaluation metrics (AUROC and accuracy score) achieved by various models utilized in several state-of-the-art approaches for the classification of hateful memes [33][34][35][36].…”