Towards generalisable hate speech detection: a review on obstacles and solutions

Yin, Wenjie; Zubiaga, Arkaitz

doi:10.7717/peerj-cs.598

Cited by 99 publications

(68 citation statements)

References 90 publications

(266 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There has also been a significant amount of criticism regarding the application of machine learning to conversation moderation, see [36] for a recent survey of the issues and challenges. The nature of online identity and social relationships, and the problems of governance are complex and involve many interacting entities with overlapping jurisdictions.…”

Section: Related Workmentioning

confidence: 99%

A New Generation of Perspective API: Efficient Multilingual Character-level Transformers

Lees¹,

Trần²,

Tay³

et al. 2022

Preprint

View full text Add to dashboard Cite

On the world wide web, toxic content detectors are a crucial line of defense against potentially hateful and offensive messages. As such, building highly effective classifiers that enable a safer internet is an important research area. Moreover, the web is a highly multilingual, cross-cultural community that develops its own lingo over time. As such, it is crucial to develop models that are effective across a diverse range of languages, usages, and styles. In this paper, we present the fundamentals behind the next version of the Perspective API from Google Jigsaw. At the heart of the approach is a single multilingual token-free Charformer model that is applicable across a range of languages, domains, and tasks. We demonstrate that by forgoing static vocabularies, we gain flexibility across a variety of settings. We additionally outline the techniques employed to make such a byte-level model efficient and feasible for productionization. Through extensive experiments on multilingual toxic comment classification benchmarks derived from real API traffic and evaluation on an array of code-switching, covert toxicity, emoji-based hate, human-readable obfuscation, distribution shift, and bias evaluation settings, we show that our proposed approach outperforms strong baselines. Finally, we present our findings from deploying this system in production.

show abstract

Section: Related Workmentioning

confidence: 99%

A New Generation of Perspective API: Efficient Multilingual Character-level Transformers

Lees¹,

Trần²,

Tay³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…However, the study of biases based on psychographic attributes (grouping individuals w.r.t their beliefs and interests) is yet to gain popularity. Note that unlike the majority of literature on toxic speech, we do not limit our toxic speech taxonomy to the demographic attributes of a group or an individual [99]. Rather, we encourage future exploration of bias categories tied with psychographic attributes.…”

Section: Categories Of Biasmentioning

confidence: 99%

“…We do not survey all the existing toxicity detection methods; instead, we focus on a subset of them exploring and mitigating bias in toxic speech detection. Meanwhile, Yin and Zubiaga [99] surveyed the literature addressing the robustness of hate speech detection methods and addressed the subject of bias in hate speech detection as well. While their discussion remained general commentary, we aim to develop an extensive understanding of these methods.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Handling Bias in Toxic Speech Detection: A Survey

Garg¹,

Masud²,

Suresh³

et al. 2022

Preprint

View full text Add to dashboard Cite

The massive growth of social media usage has witnessed a tsunami of online toxicity in teams of hate speech, abusive posts, cyberbullying, etc. Detecting online toxicity is challenging due to its inherent subjectivity. Factors such as the context of the speech, geography, socio-political climate, and background of the producers and consumers of the posts play a crucial role in determining if the content can be flagged as toxic. Adoption of automated toxicity detection models in production can lead to a sidelining of the various demographic and psychographic groups they aim to help in the first place. It has piqued researchers' interest in examining unintended biases and their mitigation. Due to the nascent and multi-faceted nature of the work, complete literature is chaotic in its terminologies, techniques, and findings. In this paper, we put together a systematic study to discuss the limitations and challenges of existing methods.We start by developing a taxonomy for categorising various unintended biases and a suite of evaluation metrics proposed to quantify such biases. We take a closer look at each proposed method for evaluating and mitigating bias in toxic speech detection. To examine the limitations of existing methods, we also conduct a case study to introduce the concept of bias shift due to knowledge-based bias mitigation methods. The survey concludes with an overview of the critical challenges, research gaps and future directions.While reducing toxicity on online platforms continues to be an active area of research, a systematic study of various biases and their mitigation strategies will help the research community produce robust and fair models.

show abstract

“…Substantial experiments by Fortuna et al [34] showed that training with one data set and testing with another one can decrease the performance by over 30%. Many potential reasons can be seen as obstacles for the generalisability [35,36,37,38] such as dataset size and annotation quality. However, little is known about their effects.…”

Section: Reliability Of Data Setsmentioning

confidence: 99%

Overview of the HASOC Track at FIRE 2020: Hate Speech and Offensive Language Identification in Tamil, Malayalam, Hindi, English and German

Mandl

Modha²,

et al. 2020

Forum for Information Retrieval Evaluation

103

View full text Add to dashboard Cite

With the growth of social media, the spread of hate speech is also increasing rapidly. Social media are widely used in many countries. Also Hate Speech is spreading in these countries. This brings a need for multilingual Hate Speech detection algorithms. Much research in this area is dedicated to English at the moment. The HASOC track intends to provide a platform to develop and optimize Hate Speech detection algorithms for Hindi, German and English. The dataset is collected from a Twitter archive and pre-classified by a machine learning system. HASOC has two sub-task for all three languages: task A is a binary classification problem (Hate and Not Offensive) while task B is a fine-grained classification problem for three classes (HATE) Hate speech, OFFENSIVE and PROFANITY. Overall, 252 runs were submitted by 40 teams. The performance of the best classification algorithms for task A are F1 measures of 0.51, 0.53 and 0.52 for English, Hindi, and German, respectively. For task B, the best classification algorithms achieved F1 measures of 0.26, 0.33 and 0.29 for English, Hindi, and German, respectively. This article presents the tasks and the data development as well as the results. The best performing algorithms were mainly variants of the transformer architecture BERT. However, also other systems were applied with good success.

show abstract

Towards generalisable hate speech detection: a review on obstacles and solutions

Cited by 99 publications

References 90 publications

A New Generation of Perspective API: Efficient Multilingual Character-level Transformers

A New Generation of Perspective API: Efficient Multilingual Character-level Transformers

Handling Bias in Toxic Speech Detection: A Survey

Overview of the HASOC Track at FIRE 2020: Hate Speech and Offensive Language Identification in Tamil, Malayalam, Hindi, English and German

Contact Info

Product

Resources

About