A Multilingual Evaluation for Online Hate Speech Detection

Abstract

The increasing popularity of social media platforms such as Twitter and Facebook has led to a rise in the presence of hate and aggressive speech on these platforms. Despite the number of approaches recently proposed in the Natural Language Processing research area for detecting these forms of abusive language, the issue of identifying hate speech at scale is still an unsolved problem. In this article, we propose a robust neural architecture that is shown to perform in a satisfactory way across different languages; namely, English, Italian, and German. We address an extensive analysis of the obtained experimental results over the three languages to gain a better understanding of the contribution of the different components employed in the system, both from the architecture point of view (i.e., Long Short Term Memory, Gated Recurrent Unit, and bidirectional Long Short Term Memory) and from the feature selection point of view (i.e., ngrams, social network–specific features, emotion lexica, emojis, word embeddings). To address such in-depth analysis, we use three freely available datasets for hate speech detection on social media in English, Italian, and German.

Resources and benchmark corpora for hate speech detection: a systematic review

“…3.3, with Twitter establishing itself as by far the most exploited source. An interesting and promising effort is that by Sabat et al (2019) and, partly, by Corazza et al (2019), who mix up textual and visual data: although still at an early stage, this path could be explored further, given the amount of image-based Table 2 it can be observed that the resources size spans from a few hundreds to several million items: this information correlates with the collection and annotation procedure inasmuch as automatic methods allow for much larger data collection, while human labeling, especially if performed by a few experts, results in smaller dataset and require a greater effort. On the other hand, if many authors prefer to collect finer-grained and higher-quality annotation on smaller samples, this suggests a commitment to creating resources of higher quality, to exploring more complex nuances and to better understand how HS can be framed with NLP techniques.…”

“…For example, our survey captures a great availability of benchmark datasets for the evaluation of abusive language and hate speech detection systems, in several languages and with several topical focuses. This adds to the challenge of investigating architectures which are stable and well-performing across different languages and abusive domains, making it a more and more promising topic to research (Corazza et al 2020;Pamungkas and Patti 2019;Ousidhoum et al 2019).…”

Time of Your Hate: The Challenge of Time in Hate Speech Detection on Social Media

“…This evaluation gap is being bridged recently by evaluation campaigns for English, Spanish (SemEval [10]), German [11], and Italian (EVALITA [12]), whose shared tasks released annotated datasets for hate speech detection. The availability of benchmarks for system evaluation and datasets for hate speech detection in different languages made the challenge of investigating architectures, which are also stable and well-performing across different languages, an exciting issue to research [13,14].…”

A Systematic Literature Review of Different Machine Learning Methods on Hate Speech Detection

“…Paper [17] is about detecting hate speech in social networks. Hate speech increased of attacks targeting spesific groups of users based on their religion, ethnicity, or social status.…”