Hate speech detection: A solved problem? The challenging case of long tail on Twitter

Abstract

In recent years, the increasing propagation of hate speech on social media and the urgent need for effective countermeasures have drawn significant investment from governments, companies, and researchers. A large number of methods have been developed for automated hate speech detection online. This aims to classify textual content into non-hate or hate speech, in which case the method may also identify the targeting characteristics (i.e., types of hate, such as race, and religion) in the hate speech. However, we notice significant difference between the performance of the two (i.e., non-hate v.s. hate). In this work, we argue for a focus on the latter problem for practical reasons. We show that it is a much more challenging task, as our analysis of the language in the typical datasets shows that hate speech lacks unique, discriminative features and therefore is found in the 'long tail' in a dataset that is difficult to discover. We then propose Deep Neural Network structures serving as feature extractors that are particularly effective for capturing the semantics of hate speech. Our methods are evaluated on the largest collection of hate speech datasets based on Twitter, and are shown to be able to outperform the best performing method by up to 5 percentage points in macro-average F1, or 8 percentage points in the more challenging case of identifying hateful content.

Towards countering hate speech against journalists on social media

“…As stated in [30] and is obvious in our dataset statistics shown in Table 4, a usual observation in hate speech datasets is their highly imbalanced nature. In imbalanced datasets, like the ones discussed in this paper, micro-averaging can inherently hide the real performance of minority classes.…”

“…The most popular network architectures are Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN). In the context of hate speech classification, CNN extracts meaningful features from word or character combinations [29,28,5,30], while RNN learns word or character dependencies in sequences of words [27,29,19,31]. Combinations of CNN and RNN models were applied in [32].…”

“…A simple Convolutional Neural Network model described in [30] acts as n-gram feature extractor. Using windows sizes of 2,3 and 4 this CNN model can extract bi-gram, tri-gram and quad-gram features.…”

“…Skipped CNN (sCNN). Extending the base CNN model in order to capture features of words that are not next to each other zhang et al [30] proposed Skipped CNN layer. Skipped CNN applies a mask to a kernel window, skipping intermediate words and associating words that are not directly near.…”

Multi-label Categorization of Accounts of Sexism using a Neural Framework

“…Badjatiya et al (2017) experiment with various deep learning approaches for the same threeway classification. Zhang and Luo (2018) explore skipped CNN and a combination of CNN and GRU for hate speech detection. Unlike these papers, we seek to categorize accounts of sexism, a specific form of discrimination or hate.…”