Detecting racism and xenophobia using deep learning models on Twitter data: CNN, LSTM and BERT

Abstract

With the growth that social networks have experienced in recent years, it is entirely impossible to moderate content manually. Thanks to the different existing techniques in natural language processing, it is possible to generate predictive models that automatically classify texts into different categories. However, a weakness has been detected concerning the language used to train such models. This work aimed to develop a predictive model based on BERT, capable of detecting racist and xenophobic messages in tweets written in Spanish. A comparison was made with different Deep Learning models. A total of five predictive models were developed, two based on BERT and three using other deep learning techniques, CNN, LSTM and a model combining CNN + LSTM techniques. After exhaustively analyzing the results obtained by the different models, it was found that the one that got the best metrics was BETO, a BERT-based model trained only with texts written in Spanish. The results of our study show that the BETO model achieves a precision of 85.22% compared to the 82.00% precision of the mBERT model. The rest of the models obtained between 79.34% and 80.48% precision. On this basis, it has been possible to justify the vital importance of developing native transfer learning models for solving Natural Language Processing (NLP) problems in Spanish. Our main contribution is the achievement of promising results in the field of racism and hate speech in Spanish by applying different deep learning techniques.

Analyzing Zero-Shot transfer Scenarios across Spanish variants for Hate Speech Detection

“…Results obtained in (Plaza-del Arco et al, 2021) showed that BETO, a monolingual LM outperforms multilingual pre-trained models such as XLM and mBERT as well as the rest of the models they evaluated for hate speech detection in Spanish. Results in line with Plaza-del Arco et al (2021) have also been achieved in other similar studies on hate speech detection (Benítez-Andrades et al, 2022;Tanase et al, 2020). Nozza (2021) studied hate speech detection against women and immigrants across three languages: Spanish, English, and Italian.…”

Enhancing ASD detection accuracy: a combined approach of machine learning and deep learning models with natural language processing

“…While the pre-training process, BERT learns to predict missing words in a sentence and to distinguish well structured information from random ones. This model is ideal for cases where named entity recognition or sentiment analysis are done among other activities [ 29 ]. So, this model is on the best candidates to give the most accurate percentage of accuracy among the other models.…”

Sentiment Analysis of Social Media Data on Ebola Outbreak Using Deep Learning Classifiers

“…To balance the computational efficiency and model accuracy, a batch size of 64 was used. The Adam optimizer was chosen to manage the update of the model weights, as it has been shown to be effective in optimizing deep learning models [ 49 , 50 ]. Additionally, a learning rate of 0.0001 was set to control the step size when performing the update, as it affects the convergence speed of the model during training.…”