Emojis as anchors to detect Arabic offensive language and hate speech

Abstract

We introduce a generic, language-independent method to collect a large percentage of offensive and hate tweets regardless of their topics or genres. We harness the extralinguistic information embedded in the emojis to collect a large number of offensive tweets. We apply the proposed method on Arabic tweets and compare it with English tweets—analyzing key cultural differences. We observed a constant usage of these emojis to represent offensiveness throughout different timespans on Twitter. We manually annotate and publicly release the largest Arabic dataset for offensive, fine-grained hate speech, vulgar, and violence content. Furthermore, we benchmark the dataset for detecting offensiveness and hate speech using different transformer architectures and perform in-depth linguistic analysis. We evaluate our models on external datasets—a Twitter dataset collected using a completely different method, and a multi-platform dataset containing comments from Twitter, YouTube, and Facebook, for assessing generalization capability. Competitive results on these datasets suggest that the data collected using our method capture universal characteristics of offensive language. Our findings also highlight the common words used in offensive communications, common targets for hate speech, specific patterns in violence tweets, and pinpoint common classification errors that can be attributed to limitations of NLP models. We observe that even state-of-the-art transformer models may fail to take into account culture, background, and context or understand nuances present in real-world data such as sarcasm.

Hate speech detection with ADHAR: a multi-dialectal hate speech corpus in Arabic

“…e best performance for Arabic was achieved by STSL, with a macro F1 score of 0.84 and a micro F1 score of 0.72. In Mubarak et al (2022), the authors introduced an Arabic multi-dialectal dataset which consists of 12,698 tweets classi ed into two main classes: clean and offensive. e offensive class was further classi ed into eight sub-classes which are: gender, race, ideology, social class, religion, disability, vulgar, and violence.…”

“…For example, Haddad et al ( 2019 ) introduced a corpus for the Tunisian dialect and Mulki et al ( 2019 ) proposed a corpus specifically for the Levantine dialect. Some other corpora include Arabic text from mixed dialects such as Albadi et al ( 2018 ), Omar et al ( 2020 ), Duwairi et al ( 2021 ), and Mubarak et al ( 2022 ). However, they lack annotation specifying the dialect of each sentence and they also lack balancing between the different dialects.…”

Arabic Toxic Tweet Classification: Leveraging the AraBERT Model

“…Althobaiti [15] proposed an automatic method for detecting offensive language and precise hate speech in Arabic tweets. They used a dataset [16] with 12,698 tweets classified into 8235 clean and 4463 offensive tweets. They also investigated the use of sentiment analysis and emoji descriptions as appending features along with the textual content of the tweets.…”

Multilingual Content Moderation: A Case Study on Reddit

“…To better understand the level of offensiveness in the content moderation task, we manually annotated 1, 238 comments (around 200 removed and 100 unremoved examples for each of the four languages) using the fine-grained taxonomy of offensiveness presented in Mubarak et al (2022), with the addition of the categories of sexuality and age. The distribution of comments for different categories is shown in Figure 3.…”