Deep-BERT: Transfer Learning for Classifying Multilingual Offensive Texts on Social Media

Wadud, Md. Anwar Hussen; Mridha, M. F.; Shin, Jungpil; Nur, Kamruddin; Saha, Aloke Kumar

doi:10.32604/csse.2023.027841

Cited by 26 publications

(4 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Random Forest (RF) is an ensemble learning technique utilized in our multilingual offensive text detection model [60]. It operates by constructing multiple decision trees during training and outputs the mode of the classes as the prediction of individual trees.…”

Section: Ensemble Methods Random Forestmentioning

confidence: 99%

Exhaustive Study into Machine Learning and Deep Learning Methods for Multilingual Cyberbullying Detection in Bangla and Chittagonian Texts

Mahmud,

Ptaszynski,

Masui

2024

Electronics

View full text Add to dashboard Cite

Cyberbullying is a serious problem in online communication. It is important to find effective ways to detect cyberbullying content to make online environments safer. In this paper, we investigated the identification of cyberbullying contents from the Bangla and Chittagonian languages, which are both low-resource languages, with the latter being an extremely low-resource language. In the study, we used both traditional baseline machine learning methods, as well as a wide suite of deep learning methods especially focusing on hybrid networks and transformer-based multilingual models. For the data, we collected over 5000 both Bangla and Chittagonian text samples from social media. Krippendorff’s alpha and Cohen’s kappa were used to measure the reliability of the dataset annotations. Traditional machine learning methods used in this research achieved accuracies ranging from 0.63 to 0.711, with SVM emerging as the top performer. Furthermore, employing ensemble models such as Bagging with 0.70 accuracy, Boosting with 0.69 accuracy, and Voting with 0.72 accuracy yielded promising results. In contrast, deep learning models, notably CNN, achieved accuracies ranging from 0.69 to 0.811, thus outperforming traditional ML approaches, with CNN exhibiting the highest accuracy. We also proposed a series of hybrid network-based models, including BiLSTM+GRU with an accuracy of 0.799, CNN+LSTM with 0.801 accuracy, CNN+BiLSTM with 0.78 accuracy, and CNN+GRU with 0.804 accuracy. Notably, the most complex model, (CNN+LSTM)+BiLSTM, attained an accuracy of 0.82, thus showcasing the efficacy of hybrid architectures. Furthermore, we explored transformer-based models, such as XLM-Roberta with 0.841 accuracy, Bangla BERT with 0.822 accuracy, Multilingual BERT with 0.821 accuracy, BERT with 0.82 accuracy, and Bangla ELECTRA with 0.785 accuracy, which showed significantly enhanced accuracy levels. Our analysis demonstrates that deep learning methods can be highly effective in addressing the pervasive issue of cyberbullying in several different linguistic contexts. We show that transformer models can efficiently circumvent the language dependence problem that plagues conventional transfer learning methods. Our findings suggest that hybrid approaches and transformer-based embeddings can effectively tackle the problem of cyberbullying across online platforms.

show abstract

Section: Ensemble Methods Random Forestmentioning

confidence: 99%

Exhaustive Study into Machine Learning and Deep Learning Methods for Multilingual Cyberbullying Detection in Bangla and Chittagonian Texts

Mahmud,

Ptaszynski,

Masui

2024

Electronics

View full text Add to dashboard Cite

show abstract

“…They explored joint multi-lingual and jointtranslation approaches and achieved the best performance with the translation-based method (Arabic-BERT). Later, [32] addressed the offensive language detection problem using the MTC framework for English and Bengali languages. The authors proposed a Deep-BERT model and obtained effective performance.…”

Section: B Mtc Approachesmentioning

confidence: 99%

“…64, and 128. Three batch sizes (16,32,64) are evaluated one by one to investigate the impact of each batch size. Likewise, three learning rates are explored to see their impact on the training, validation, and test part of the multi-lingual dataset.…”

Section: E Fine-tunningmentioning

confidence: 99%

Fine-Tuning Transformer Models Using Transfer Learning for Multilingual Threatening Text Identification

Rehan,

Malik,

Jamjoom

2023

IEEE Access

View full text Add to dashboard Cite

Threatening content detection on social media has recently gained attention. There is very limited work regarding threatening content detection in low-resource languages, especially in Urdu. Furthermore, previous work explored only mono-lingual approaches, and multi-lingual threatening content detection was not studied. This research addressed the task of Multi-lingual Threatening Content Detection (MTCD) in Urdu and English languages by exploiting transfer learning methodology with fine-tuning techniques. To address the multi-lingual task, we investigated two methodologies: 1) Joint multi-lingual, and 2) Joint-translated method. The former approach employs the concept of building a universal classifier for different languages whereas the latter approach applies the translation process to transform the text into one language and then perform classification. We explore the Multilingual Representations for Indian Languages (MuRIL) and Robustly Optimized BERT Pre-Training Approach (RoBERTa) with fine-tuning that already demonstrated state-of-the-art in capturing the contextual and semantic characteristics within the text. For hyper-parameters, manual search and grid search strategies are utilized to find the optimum values. Various experiments are performed on bi-lingual English and Urdu datasets and findings revealed that the proposed methodology outperformed the baselines and showed benchmark performance. The RoBERTa model achieved the highest performance by demonstrating 92% accuracy and 90% macro f1-score with the joint multi-lingual approach.

show abstract

“…Deep learning has achieved excellent performance in natural language processing [1,2] and computer vision [3][4][5]. Therefore, many recognition algorithms based on deep learning for interference signals were proposed to solve the problems of traditional interference signal recognition algorithms whose accuracy is low and significantly affected by artificial feature selection [6][7][8][9][10][11][12].…”

Section: Introductionmentioning

confidence: 99%

Contrastive Clustering for Unsupervised Recognition of Interference Signals

Chen¹,

Zhao²,

Ye³

et al. 2023

Computer Systems Science and Engineering

View full text Add to dashboard Cite

Interference signals recognition plays an important role in anti-jamming communication. With the development of deep learning, many supervised interference signals recognition algorithms based on deep learning have emerged recently and show better performance than traditional recognition algorithms. However, there is no unsupervised interference signals recognition algorithm at present. In this paper, an unsupervised interference signals recognition method called double phases and double dimensions contrastive clustering (DDCC) is proposed. Specifically, in the first phase, four data augmentation strategies for interference signals are used in data-augmentation-based (DA-based) contrastive learning.In the second phase, the original dataset's k-nearest neighbor set (KNNset) is designed in double dimensions contrastive learning. In addition, a dynamic entropy parameter strategy is proposed. The simulation experiments of 9 types of interference signals show that random cropping is the best one of the four data augmentation strategies; the feature dimensional contrastive learning in the second phase can improve the clustering purity; the dynamic entropy parameter strategy can improve the stability of DDCC effectively. The unsupervised interference signals recognition results of DDCC and five other deep clustering algorithms show that the clustering performance of DDCC is superior to other algorithms. In particular, the clustering purity of our method is above 92%, SCAN's is 81%, and the other three methods' are below 71% when jammingnoise-ratio (JNR) is −5 dB. In addition, our method is close to the supervised learning algorithm.

show abstract

Deep-BERT: Transfer Learning for Classifying Multilingual Offensive Texts on Social Media

Cited by 26 publications

References 28 publications

Exhaustive Study into Machine Learning and Deep Learning Methods for Multilingual Cyberbullying Detection in Bangla and Chittagonian Texts

Exhaustive Study into Machine Learning and Deep Learning Methods for Multilingual Cyberbullying Detection in Bangla and Chittagonian Texts

Fine-Tuning Transformer Models Using Transfer Learning for Multilingual Threatening Text Identification

Contrastive Clustering for Unsupervised Recognition of Interference Signals

Contact Info

Product

Resources

About