Hate Speech Detection in Twitter using Transformer Methods

Abstract

Social media networks such as Twitter are increasingly utilized to propagate hate speech while facilitating mass communication. Recent studies have highlighted a strong correlation between hate speech propagation and hate crimes such as xenophobic attacks. Due to the size of social media and the consequences of hate speech in society, it is essential to develop automated methods for hate speech detection in different social media platforms. Several studies have investigated the application of different machine learning algorithms for hate speech detection. However, the performance of these algorithms is generally hampered by inefficient sequence transduction. The Vanilla recurrent neural networks and recurrent neural networks with attention have been established as state-of-the-art methods for the assignments of sequence modeling and sequence transduction. Unfortunately, these methods suffer from intrinsic problems such as long-term dependency and lack of parallelization. In this study, we investigate a transformer-based method and tested it on a publicly available multiclass hate speech corpus containing 24783 labeled tweets. DistilBERT transformer method was compared against attention-based recurrent neural networks and other transformer baselines for hate speech detection in Twitter documents. The study results show that DistilBERT transformer outperformed the baseline algorithms while allowing parallelization.

Twitter Hate Speech Detection: A Systematic Review of Methods, Taxonomy Analysis, Challenges, and Opportunities

“…The introduction of linguistic features might be a promising path [45,59]. A further improvement to this task could be to incorporate textual and image data together in the detection models [12,37,74]. For possible work corresponding to the hate speech task, the examination of platforms other than Twitter that fertilizes hate speech could be explored, as suggested by [86].…”

“…Nevertheless, they suffered from ineffective sequence acodingly representation. As such, the transformer-based method was provided by [74]. In [86], researchers noted that hate speech can be directed at a single person or a group of people (generalized).…”

Multimodal detection of hateful memes by applying a vision-language pre-training model

“…Though earlier methods have been limited in their usefulness as they often can only handle obvious patterns, encouragingly, the present transformer-based deep learning models have reached a much higher accuracy (Acc.) of more than 90% in detecting textual hateful messages due to its powerful capacity in the semantic comprehension [ 17 ]. The commercial application of deep learning models effectively reduces the spreading of textual hateful messages.…”

“…Moreover, content moderators are known to suffer post-traumatic stress disorder (PTSD)-like syndromes after repetitively reviewing violent and exploitative content [9]. As a result, generations of automatic hateful message detection methods have been developed since 2009, from the initial block-word-list approach to the current deep learning-based technology [10][11][12][13][14][15][16][17]. Though earlier methods have been limited in their usefulness as they often can only handle obvious patterns, encouragingly, the present transformer-based deep learning models have reached a much higher accuracy (Acc.)…”

Application of Transformer-Based Language Models to Detect Hate Speech in Social Media

“…The authors, focusing exclusively on the Transformer-based RoBERTa, demonstrated that large pretrained Transformer LMs are highly potent in classification tasks even when the dataset size is very small as compared to most other deep learning applications. Mutanga et al (2020) also evaluated an ensemble of Transformer models, focusing mainly on the DistilBERT model, which is a scaled-down version of the original BERT with increased speed while maintaining considerable consistency in performance.…”