Overview of the HASOC Subtrack at FIRE 2022: Hate Speech and Offensive Content Identification in English and Indo-Aryan Languages

Satapara, Shrey; Majumder, Prasenjit; Mandl, Thomas; Modha, Sandip; Madhu, Hiren; Ranasinghe, Tharindu; Zampieri, Marcos; North, Kai; Damith, Premasiri,

doi:10.1145/3574318.3574326

Cited by 12 publications

(5 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some studies used this dataset to carry out experiments [24] [25][26]. Similarly, conversational hate speech detection in code-mixed language dataset is available in Hindi and English language [14]. The dataset was collected from Twitter.…”

Section: Related Workmentioning

confidence: 99%

TABHATE: A Target-based Hate Speech Detection Dataset in Hindi

Sharma

Singh

Gupta

2023

Preprint

View full text Add to dashboard Cite

Social media has over the years provided a medium for creation and dissemination of opinions and thoughts through online platforms. While it allows users to express their views, sentiments and emotions, some people try to use it to generate and share unpleasant and hateful content. Such content is now referred to as hate speech and it may target an individual, a group, a community, or a country. During the last few years, several techniques have been developed to automatically detect and identify hate speech, offensive and abusive content from social media platforms. However, majority of the studies focused on hate speech detection in English language texts. With social media getting higher penetration across different geographies, there is now a significant amount of content generated in various languages. Though there have been significant advancements in algorithmic approaches for the task, the non-availability of suitable dataset in other languages poses a problem in research advancement in them. Hindi is one such widely spoken language where such datasets are not available. This work attempts to bridge this research gap by presenting a curated and annotated dataset for target-based hate speech (TABHATE) in the Hindi language. The dataset comprises of 2,020 tweets and is annotated by three independent annotators. A multiclass labelling is used where each tweet is labelled as: (i) individual targeting, (ii) community targeting, and (iii) none. Inter annotator agreement is computed. The suitability of dataset is then further explored by applying some standard deep learning and transformer-based models for the task of hate speech detection. The experimental results obtained show that the dataset can be used for experimental work on hate speech detection of Hindi language texts.

show abstract

Section: Related Workmentioning

confidence: 99%

TABHATE: A Target-based Hate Speech Detection Dataset in Hindi

Sharma

Singh

Gupta

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…HASOC 2022 featured three tasks (Satapara et al 2022). Task 1 was a continuation of the 2021 task 2, where the goal was to detect hate and offensive content in conversations (Modha et al 2022) where the classes were hate offensive and non-hate offensive.…”

Section: Hasoc: Hate Speech and Offensive Content Identification In E...mentioning

confidence: 99%

OffensEval 2023: Offensive language identification in the age of Large Language Models

Zampieri,

Rosenthal,

Nakov

et al. 2023

Nat. Lang. Eng.

Self Cite

View full text Add to dashboard Cite

The OffensEval shared tasks organized as part of SemEval-2019–2020 were very popular, attracting over 1300 participating teams. The two editions of the shared task helped advance the state of the art in offensive language identification by providing the community with benchmark datasets in Arabic, Danish, English, Greek, and Turkish. The datasets were annotated using the OLID hierarchical taxonomy, which since then has become the de facto standard in general offensive language identification research and was widely used beyond OffensEval. We present a survey of OffensEval and related competitions, and we discuss the main lessons learned. We further evaluate the performance of Large Language Models (LLMs), which have recently revolutionalized the field of Natural Language Processing. We use zero-shot prompting with six popular LLMs and zero-shot learning with two task-specific fine-tuned BERT models, and we compare the results against those of the top-performing teams at the OffensEval competitions. Our results show that while some LMMs such as Flan-T5 achieve competitive performance, in general LLMs lag behind the best OffensEval systems.

show abstract

“…Hatespeech, trolling, cyberaggression and cyberbullying have become the focal areas of regular shared tasks, conferences and special issues (Zampieri et al, , 2019bSatapara et al, 2023;Modha et al, 2022). There have also been recent works dedicated to the detection of mental health problems online, such as on social media (Bucur et al, 2021;Bannink et al, 2014).…”

Section: Nlp For Safeguardingmentioning

confidence: 99%

Deep Learning Approaches to Detecting Safeguarding Concerns in Schoolchildren’s Online Conversations

Franklin,

Ranasinghe

2023

Proceedings of the Conference Recent Advances in Natural Language Processing - Large Language Models for Natural Language Proce

View full text Add to dashboard Cite

For school teachers and Designated Safeguarding Leads (DSLs), computers and other schoolowned communication devices are both indispensable and deeply worrisome. For their education, children require access to the internet, as well as a standard institutional ICT infrastructure, including e-mail and other forms of online communication technology. Given the sheer volume of data being generated and shared on a daily basis within schools, most teachers and DSLs can no longer monitor the safety and wellbeing of their students without the use of specialist safeguarding software. In this paper, we experiment with the use of stateof-the-art neural network models on the modelling of a dataset of almost 9,000 anonymised child-generated chat messages on the Microsoft Teams platform. The dataset was manually annotated into two binary classes: true positives (real safeguarding concerns) and false positives (false alarms) that a monitoring program would be interested in. These classes were then further annotated into eight fine-grained classes of safeguarding concerns (or false alarms). For the binary classification, we achieved a macro F1 score of 87.32, while for the fine-grained classification, our models achieved a macro F1 score of 73.56. This first experiment into the use of Deep Learning for detecting safeguarding concerns represents an important step towards achieving high-accuracy and reliable monitoring information for busy teachers and safeguarding leads.

show abstract

Overview of the HASOC Subtrack at FIRE 2022: Hate Speech and Offensive Content Identification in English and Indo-Aryan Languages

Cited by 12 publications

References 25 publications

TABHATE: A Target-based Hate Speech Detection Dataset in Hindi

TABHATE: A Target-based Hate Speech Detection Dataset in Hindi

OffensEval 2023: Offensive language identification in the age of Large Language Models

Deep Learning Approaches to Detecting Safeguarding Concerns in Schoolchildren’s Online Conversations

Contact Info

Product

Resources

About