Benchmark dataset of memes with text transcriptions for automatic detection of multi-modal misogynistic content

Gasparini, Francesca; Rizzi, Giulia; Saibene, Aurora; Fersini, Elisabetta

doi:10.48550/arxiv.2106.08409

Cited by 4 publications

(8 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…I.B: Misogynistic/Sexist: Misogyny and sexism against women have grown a foothold within social media communities, reinvigorating age-old patriarchal establishments of baseless name-calling, objectifying their appearances, and stereotyping gender roles, which has been explored in the literature [Gasparini et al, 2021]. This is especially fueled by the cryptic use of sexism disguised as humor via memes.…”

Section: Types Of Harmful Memes I: Hatementioning

confidence: 99%

“…Several studies have focused on content and implicit offensive analogies within memes. Some leveraged unimodal [Giri et al, 2021] and multimodal information [Suryawanshi et al, 2020a], and investigating simple encoder and early fusion strategies for classifying offensive memes, while using techniques such as stacked LSTM/BiLSTM/CNN (Text) along with VGG-16 [Simonyan and Zisserman, 2015] to model multimodality, ultimately achieving an F1 score of 0.71 and accuracy of 0.50. To address contextualization, [Shang et al, 2021b] used analogyaware multimodality, by combining ResNet50 [He et al, 2016] and GloVe-based LSTM, and attentive multimodal analogy alignment via supervised learning, while incorporating contextual discourse, yielding 0.72 and 0.69 accuracy for Redditand Gab-based datasets, respectively.…”

Section: Ii: Offensive Memesmentioning

confidence: 99%

“…• Complex abstraction: Memes can efficiently abstract away complex ideas using creative and powerful customization of visual and linguistic nuances. At the same time, memes with overlapping snippets, patterned text and irony, sarcasm or implicit anti-semitism are non-trivial [Chandra et al, 2021]. For instance, the subtle use of triple parentheses in memes can insinuate a targeted entity whilst underlining an anti-semitic narrative [Tuters and Hagen, 2020].…”

Section: Major Challengesmentioning

confidence: 99%

See 2 more Smart Citations

Detecting and Understanding Harmful Memes: A Survey

Teeti

Khan

Shahbaz

et al. 2022

Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

This survey targets intention and trajectory prediction in Autonomous Vehicles (AV), as AV companies compete to create dedicated prediction pipelines to avoid collisions. The survey starts with a formal definition of the prediction problem and highlights its challenges, to then critically compare the models proposed in the last 2-3 years in terms of how they overcome these challenges. Further, it lists the latest methodological and technical trends in the field and comments on the efficacy of different machine learning blocks in modelling various aspects of the prediction problem. It also summarises the popular datasets and metrics used to evaluate prediction models, before concluding with the possible research gaps and future directions.

show abstract

Section: Types Of Harmful Memes I: Hatementioning

confidence: 99%

Section: Ii: Offensive Memesmentioning

confidence: 99%

Section: Major Challengesmentioning

confidence: 99%

See 1 more Smart Citation

Detecting and Understanding Harmful Memes: A Survey

Teeti

Khan

Shahbaz

et al. 2022

Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

show abstract

“…misogynistic-meme. An expert-labeled open misogynistic dataset (Gasparini et al, 2021), it contains 800 memes with manually transcribed text, the misogynisticDE field is used as the label for misogyny.…”

Section: Training Datasetsmentioning

confidence: 99%

SRCB at SemEval-2022 Task 5: Pretraining Based Image to Text Late Sequential Fusion System for Multimodal Misogynous Meme Identification

Zhang¹,

Wang²

2022

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)

View full text Add to dashboard Cite

Online misogyny meme detection is an image/text multimodal classification task, the complicated relation of image and text challenges the intelligent system's modality fusion learning capability. In this paper, we investigate the single-stream UNITER and dual-stream CLIP multimodal pretrained models on their capability to handle strong and weakly correlated image/text pairs. The XGBoost classifier with image features extracted by the CLIP model has the highest performance and being robust on domain shift. Based on this, we propose the PBR system, an ensemble system of Pretraining models, Boosting method and Rule-based adjustment, text information is fused into the system using our late sequential fusion scheme. Our system ranks 1st place on both sub-task A and sub-task B of the SemEval-2022 Task 5 Multimedia Automatic Misogyny Identification, with 0.834/0.731 macro F1 scores for subtask A/B correspondingly.

show abstract

“…Multi-modal hate speech detection had less attention in the research literature than traditional text-only methods. In the past two years, several datasets and challenges have addressed this by proposing detection tasks on meme-based data (Kiela et al, 2020;Gasparini et al, 2021;Miliani et al, 2020). Misogyny detection, as a subgroup of hate speech detection tasks, has also been more frequently encountered in research in recent years.…”

Section: Introductionmentioning

confidence: 99%

UPB at SemEval-2022 Task 5: Enhancing UNITER with Image Sentiment and Graph Convolutional Networks for Multimedia Automatic Misogyny Identification

Paraschiv¹,

Dascălu²,

Cercel³

2022

Preprint

View full text Add to dashboard Cite

In recent times, the detection of hate-speech, offensive, or abusive language in online media has become an important topic in NLP research due to the exponential growth of social media and the propagation of such messages, as well as their impact. Misogyny detection, even though it plays an important part in hatespeech detection, has not received the same attention. In this paper, we describe our classification systems submitted to the SemEval-2022 Task 5: MAMI -Multimedia Automatic Misogyny Identification. The shared task aimed to identify misogynous content in a multi-modal setting by analysing meme images together with their textual captions. To this end, we propose two models based on the pre-trained UNITER model, one enhanced with an image sentiment classifier, whereas the second leverages a Vocabulary Graph Convolutional Network (VGCN). Additionally, we explore an ensemble using the aforementioned models. Our best model reaches an F1-score of 71.4% in Sub-task A and 67.3% for Subtask B positioning our team in the upper third of the leaderboard. We release the code and experiments for our models on GitHub 1 .

show abstract

Benchmark dataset of memes with text transcriptions for automatic detection of multi-modal misogynistic content

Cited by 4 publications

References 9 publications

Detecting and Understanding Harmful Memes: A Survey

Detecting and Understanding Harmful Memes: A Survey

SRCB at SemEval-2022 Task 5: Pretraining Based Image to Text Late Sequential Fusion System for Multimodal Misogynous Meme Identification

UPB at SemEval-2022 Task 5: Enhancing UNITER with Image Sentiment and Graph Convolutional Networks for Multimedia Automatic Misogyny Identification

Contact Info

Product

Resources

About