Towards multidomain and multilingual abusive language detection: a survey

Pamungkas, Endang Wahyu; Basile, Valerio; Patti, Viviana

doi:10.1007/s00779-021-01609-1

Cited by 23 publications

(31 citation statements)

References 117 publications

(261 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Applying our methodology to other languages is not trivial, as it depends on the availability of language resources and robust NLP tools for them (Pamungkas et al, 2021). Fortunately, full-fledged NLP pipelines do exist for many languages, thanks for instance to large-scale initiatives such as Universal Dependencies, which provides among its deliverables the UDpipe software library and a broad set of trained models in more than 70 languages (Nivre et al, 2016;Straka et al, 2016).…”

Section: Discussionmentioning

confidence: 99%

Investigating the role of swear words in abusive language detection tasks

Pamungkas

Basile

Patti

2022

Lang Resources & Evaluation

Self Cite

View full text Add to dashboard Cite

Swearing plays an ubiquitous role in everyday conversations among humans, both in oral and textual communication, and occurs frequently in social media texts, typically featured by informal language and spontaneous writing. Such occurrences can be linked to an abusive context, when they contribute to the expression of hatred and to the abusive effect, causing harm and offense. However, swearing is multifaceted and is often used in casual contexts, also with positive social functions. In this study, we explore the phenomenon of swearing in Twitter conversations, by automatically predicting the abusiveness of a swear word in a tweet as the main investigation perspective. We developed the Twitter English corpus SWAD (Swear Words Abusiveness Dataset), where abusive swearing is manually annotated at the word level. Our collection consists of 2577 instances in total from two phases of manual annotation. We developed models to automatically predict abusive swearing, to provide an intrinsic evaluation of SWAD and confirm the robustness of the resource. We model this prediction task as three different tasks, namely sequence labeling, text classification, and target-based swear word abusiveness prediction. We experimentally found that our intention to model the task similarly to aspect-based sentiment analysis leads to promising results. Subsequently, we employ the classifier to improve the prediction of abusive language in several standard benchmarks. The results of our experiments show that additional abusiveness feature of the swear words is able to improve the performance of abusive language detection models in several benchmark datasets.

show abstract

Section: Discussionmentioning

confidence: 99%

Investigating the role of swear words in abusive language detection tasks

Pamungkas

Basile

Patti

2022

Lang Resources & Evaluation

Self Cite

View full text Add to dashboard Cite

show abstract

“…Hate speech datasets also differ in annotation schema, which is shown in recent surveys Vidgen and Derczynski (2020); Poletto et al (2021); Pamungkas et al (2021a). This variety is due to the multifaceted nature of hate speech, as it can be directed against individuals or groups, be implicit or explicit, and have varying themes such as race, gender, or disability.…”

Section: Hate Speech Definitionsmentioning

confidence: 99%

“…2 . Considering the above discussed variance issues of hate speech definition and label sets, multilingual hate speech detection remains an important and relevant task, since social media platforms are multilingual spaces where people may easily communicate in their native tongue Pamungkas et al (2021a). Due to the costliness of collecting and annotating new data, it is relevant to consider ways of exploiting resources that are already available.…”

Section: Hate Speech Data Scarcity and Cross-lingual Transfermentioning

confidence: 99%

Label modification and bootstrapping for zero-shot cross-lingual hate speech detection

Bigoulaeva

Hangya²,

Gurevych

et al. 2023

Lang Resources & Evaluation

View full text Add to dashboard Cite

The goal of hate speech detection is to filter negative online content aiming at certain groups of people. Due to the easy accessibility and multilinguality of social media platforms, it is crucial to protect everyone which requires building hate speech detection systems for a wide range of languages. However, the available labeled hate speech datasets are limited, making it difficult to build systems for many languages. In this paper we focus on cross-lingual transfer learning to support hate speech detection in low-resource languages, while highlighting label issues across application scenarios, such as inconsistent label sets of corpora or differing hate speech definitions, which hinder the application of such methods. We leverage cross-lingual word embeddings to train our neural network systems on the source language and apply them to the target language, which lacks labeled examples, and show that good performance can be achieved. We then incorporate unlabeled target language data for further model improvements by bootstrapping labels using an ensemble of different model architectures. Furthermore, we investigate the issue of label imbalance in hate speech datasets, since the high ratio of non-hate examples compared to hate examples often leads to low model performance. We test simple data undersampling and oversampling techniques and show their effectiveness.

show abstract

“…OffensEval 2020 (Zampieri et al 2020) featured offensive language identification datasets in Arabic, Danish, Greek, Turkish and English. We direct interested readers to relevant surveys for further information (Schmidt and Wiegand 2017;Fortuna and Nunes 2018;Poletto et al 2020;Vidgen and Derczynski 2020;Pamungkas, Basile, and Patti 2021b). Only a handful of studies have investigated zero-shot cross-lingual transfer learning for hate speech detection.…”

Section: Related Workmentioning

confidence: 99%

Improving Zero-Shot Cross-Lingual Hate Speech Detection with Pseudo-Label Fine-Tuning of Transformer Language Models

Zia

Castro

Zubiaga

et al. 2022

ICWSM

View full text Add to dashboard Cite

Hate speech has proliferated on social media platforms in recent years. While this has been the focus of many studies, most works have exclusively focused on a single language, generally English. Low-resourced languages have been neglected due to the dearth of labeled resources. These languages, however, represent an important portion of the data due to the multilingual nature of social media. This work presents a novel zero-shot, cross-lingual transfer learning pipeline based on pseudo-label fine-tuning of Transformer Language Models for automatic hate speech detection. We employ our pipeline on benchmark datasets covering English (source) and 6 different non-English (target) languages written in 3 different scripts. Our pipeline achieves an average improvement of 7.6% (in terms of macro-F1) over previous zero-shot, cross-lingual models. This demonstrates the feasibility of high accuracy automatic hate speech detection for low-resource languages. We release our code and models at https://github.com/harisbinzia/ZeroshotCrosslingualHateSpeech.

show abstract

Towards multidomain and multilingual abusive language detection: a survey

Cited by 23 publications

References 117 publications

Investigating the role of swear words in abusive language detection tasks

Investigating the role of swear words in abusive language detection tasks

Label modification and bootstrapping for zero-shot cross-lingual hate speech detection

Improving Zero-Shot Cross-Lingual Hate Speech Detection with Pseudo-Label Fine-Tuning of Transformer Language Models

Contact Info

Product

Resources

About