Pay “Attention” to your Context when Classifying Abusive Language

Abstract

The goal of any social media platform is to facilitate healthy and meaningful interactions among its users. But more often than not, it has been found that it becomes an avenue for wanton attacks. We propose an experimental study that has three aims: 1) to provide us with a deeper understanding of current datasets that focus on different types of abusive language, which are sometimes overlapping (racism, sexism, hate speech, offensive language and personal attacks); 2) to investigate what type of attention mechanism (contextual vs. self-attention) is better for abusive language detection using deep learning architectures; and 3) to investigate whether stacked architectures provide an advantage over simple architectures for this task.

Self-Contextualized Attention for Abusive Language Identification

“…Centering the analysis of results on the first three baselines and on our classification framework (columns 2 -5), the results indicate that the use of AM outperformed the base Bi-GRU network (column 2 vs columns 3 -5) by at least a margin of 1.1%. In addition, the use of the CA outperformed the use of SA (column 4 vs column 3) by at least a margin of 1.2%, which is consistent according to the results obtained in (Chakrabarty et al, 2019). Finally, comparing the use of our proposed SCA mechanism against the use of SA and CA (column 5 vs columns 3 and 4), better results are obtained in the four evaluation datasets, improving the results by at least a margin of 1.1%.…”

“…One of the first works introducing attention into the task used the SA mechanism to detect abuse in portal news and Wikipedia (Pavlopoulos et al, 2017). Subsequently, (Chakrabarty et al, 2019) showed that the use of CA introduced by (Yang et al, 2016) improved the results of SA in this task. Later in (Jarquín- the use of the CA is extended at a word n-grams level, showing the advantages in the usage of word sequences when identifying AL.…”

“…In order to integrate our proposed SCA mechanism into the AL identification task, we adapt a modular and well-performing DNN architecture, as our classification framework. This architecture was presented in (Yang et al, 2016;Chakrabarty et al, 2019) and its designed to modularly manage different AM. The adapted architecture is shown in Figure 2; it consists of four main stages, which are described below.…”

“…This indicates that the importance and interpretation of each word is highly context dependent, and, accordingly, this particular issue evidences one of the reasons why traditional bag-of-words methods tend to generate many false positives in their predictions. Few works related to this task have explored the importance of words according to their context; particularly, the use of Deep Learning (DL) approaches with the addition of the Attention Mechanism (AM) has been explored as an alternative to solve this issue (Pavlopoulos et al, 2017;Chakrabarty et al, 2019;.…”

HurtBERT: Incorporating Lexical Features with BERT for the Detection of Abusive Language

“…Most works propose variations on neural architectures such as Recurrent Neural Networks (especially Long Shortterm Memory networks), or Convolutional Neural Networks (Mishra et al, 2019). An investigation on what type of attention mechanism (contextual vs. self-attention) is better for abusive language detection using deep learning architectures is proposed in (Chakrabarty et al, 2019). Character-based models have also been proposed for this task .…”