Overview of the HASOC Subtrack at FIRE 2021: Hate Speech and Offensive Content Identification in English and Indo-Aryan Languages and Conversational Hate Speech

Abstract

The widespread of offensive content online such as hate speech poses a growing societal problem. AI tools are necessary for supporting the moderation process at online platforms. For the evaluation of these identification tools, continuous experimentation with data sets in different languages are necessary. The HASOC track (Hate Speech and Offensive Content Identification) is dedicated to develop benchmark data for this purpose. This paper presents the HASOC subtrack for English, Hindi, and Marathi. The data set was assembled from Twitter. This subtrack has two sub-tasks. Task A is a binary classification problem (Hate and Not Offensive) offered for all three languages. Task B is a fine-grained classification problem for three classes (HATE) Hate speech, OFFENSIVE and PROFANITY offered for English and Hindi. Overall, 652 runs were submitted by 65 teams. The performance of the best classification algorithms for task A are F1 measures 0.91, 0.78 and 0.83 for Marathi, Hindi and English, respectively. This overview presents the tasks and the data development as well as the detailed results. The systems submitted to the competition applied a variety of technologies. The best performing algorithms were mainly variants of transformer architectures.

A simple language-agnostic yet very strong baseline system for hate speech and offensive content identification

“…Its performance, averaged on the two Hindi problems and the Marathi problem, ranks it in first place among the teams that proposed systems for at least two of these problems. These performances suggest that it is an interesting reference level to evaluate the benefits of using more complex approaches that are frequently used to address this type of task such as deep learning or taking into account complementary resources (Mandl et al, 2019;Mandl et al, 2020;. However, it is essential to note that the proposed system never ranked first in any specific task.…”

“…Not surprisingly, a lot of research is being done to develop automatic detection systems. As in many NLP domains, deep learning approaches and the use of pre-computed embeddings have proven to be the most efficient, even in languages with few resources (Mandl et al, 2019;Mandl et al, 2020). However, traditional machine learning systems have sometimes proven to be very competitive (Mujadia et al, 2019;Saroj et al, 2019).…”

Multilingual Hate Speech and Offensive Content Detection using Modified Cross-entropy Loss

“…The Hate speech and Offensive Content Identification in English and Indo-Aryan Languages HASOC 2021 [5,6] purposes two different tasks, in 3 different languages English, Hindi, Marathi. The authors participated in both tasks for English and Hindi languages.…”

“…The authors would like to thank the organizers of Hate Speech and Offensive Content Identification in Indo-Aryan Languages 2021 [5] for conducting this data challenge. The authors gratefully acknowledge google colab for providing GPU's to do the computation.…”

“…This makes the classification task additionally more difficult for most progressive frameworks. HASOC 2021 [5] is a shared task for the identification of hateful and offensive content in English and Indo-Aryan Languages. We participated in two sub-tasks for English and Hindi language [6].…”