Alpha at SemEval-2021 Task 6: Transformer Based Propaganda Classification

Abstract

This paper describes our system participated in Task 6 of SemEval-2021: this task focuses on multimodal propaganda technique classification and it aims to classify given image and text into 22 classes. In this paper, we propose to use transformer-based (Vaswani et al., 2017) architecture to fuse the clues from both image and text. We explore two branches of techniques including fine-tuning the text pre-trained transformer with extended visual features and fine-tuning the multimodal pre-trained transformers. For the visual features, we experiment with both grid features extracted from ResNet(He et al., 2016) network and salient region features from a pretrained object detector. Among the pre-trained multimodal transformers, we choose ERNIE-ViL (Yu et al., 2020), a two-steam crossattended transformers model pre-trained on large-scale image-caption aligned data. Finetuning ERNIE-ViL for our task produces a better performance due to general joint multimodal representation for text and image learned by ERNIE-ViL. Besides, as the distribution of the classification labels is extremely unbalanced, we also make a further attempt on the loss function and the experiment results show that focal loss would perform better than cross-entropy loss. Lastly, we ranked first place at sub-task C in the final competition. * indicates equal contribution.

UniBO at SemEval-2022 Task 5: A Multimodal bi-Transformer Approach to the Binary and Fine-grained Identification of Misogyny in Memes

“…Dimitrov et al (2021) proposed a shared task on the identification of propaganda techniques. Feng et al (2021) approached it with a pre-trained transformer using text with visual features. They extracted grid features using ResNet50 (He et al, 2016) and salient region features using BUTD (Anderson et al, 2018).…”

Emotion-Enhanced Multi-Modal Persuasive Techniques Detection Using Split Features

“…Dimitrov et al [6] annotated both the text of the memes, as well as the techniques appearing in the visual content of the memes. As for the task of identify which persuasive techniques are used in the text and the image, Alpha [3], MinD [4] and 1213Li [10] are the top three best performing algorithms. Alpha [3] built an ensemble of fine-tuned De-BERTA+ResNet, DeBERTA+BUTD, and ERNIEVIL systems.…”

Faking Fake News for Real Fake News Detection: Propaganda-loaded Training Data Generation

“…(2019) present a fragment-level propaganda detection dataset, where specific propaganda techniques are labeled onto spans of text instead of each document. Recent approaches for detecting these propaganda techniques rely on pre-trained transformers (Morishita et al, 2020;Chernyavskiy et al, 2020;Feng et al, 2021;Tian et al, 2021). By contrast, we focus on detecting disinformative articles with propaganda signals.…”