Bridging the Gaps: Multi Task Learning for Domain Transfer of Hate Speech Detection

Abstract

Hate speech detection has been the subject of high research attention, due to the scale of content created on social media. In spite of the attention and the sensitive nature of the task, privacy preservation in hate speech detection has remained under-studied. The majority of research has focused on centralised machine learning infrastructures which risk leaking data. In this paper, we show that using federated machine learning can help address privacy the concerns that are inherent to hate speech detection while obtaining up to 6.81% improvement in terms of F1-score.

A BERT-Based Transfer Learning Approach for Hate Speech Detection in Online Social Media

“…Table 2 summarized the obtained results for fine-tuning strategies along with the official baselines. We use Waseem and Hovy [22], Davidson et al [3], and Waseem et al [23] as baselines and compare the results with our different fine-tuning strategies using pre-trained BERT base model. The evaluation results are reported on the test dataset and on three different metrics: precision, recall, and weighted-average F1-score.…”

“…Zhang et al [25] used a CNN+GRU (Gated Recurrent Unit network) neural network model initialized with pre-trained word2vec embeddings to capture both word/character combinations (e. g., n-grams, phrases) and word/character dependencies (order information). Waseem et al [23] brought a new insight to hate speech and abusive language detection tasks by proposing a multi-task learning framework to deal with datasets across different annotation schemes, labels, or geographic and cultural influences from data sampling. Founta et al [7] built a unified classification model that can efficiently handle different types of abusive language such as cyberbullying, hate, sarcasm, etc.…”

“…To extend this dataset, Waseem [20] also provided another dataset containing 6.9k of tweets annotated with both expert and crowdsourcing users as racism, sexism, neither, or both. Since both datasets are overlapped partially and they used the same strategy in definition of hateful content, we merged these two datasets following Waseem et al [23] to make our imbalance data a bit larger. Davidson et al [3] used the Twitter API to accumulate 84.4 million tweets from 33,458 twitter users containing particular terms from a pre-defined lexicon of hate speech words and phrases, called Hatebased.org.…”

“…To detect online hate speech, a large number of scientific studies have been dedicated by using Natural Language Processing (NLP) in combination with Machine Learning (ML) and Deep Learning (DL) methods [1,8,11,13,22,25]. Although supervised machine learning-based approaches have used different text mining-based features such as surface features, sentiment analysis, lexical resources, linguistic features, knowledge-based features or user-based and platformbased metadata [3,6,23], they necessitate a well-defined feature extraction approach. The trend now seems to be changing direction, with deep learning models being used for both feature extraction and the training of classifiers.…”

Racial Bias in Hate Speech and Abusive Language Detection Datasets

“…We expect that this result occurred for two reasons. First, the dataset contains a large number of cases where AAE is used (Waseem et al, 2018). Second, many of the AAE tweets also use words like "n*gga" and "b*tch", and are thus frequently associated with the hate speech and offensive classes, resulting in "false positive bias" (Dixon et al, 2018).…”