How to detect propaganda from social media? Exploitation of semantic and fine-tuned language models

Abstract

Online propaganda is a mechanism to influence the opinions of social media users. It is a growing menace to public health, democratic institutions, and public society. The present study proposes a propaganda detection framework as a binary classification model based on a news repository. Several feature models are explored to develop a robust model such as part-of-speech, LIWC, word uni-gram, Embeddings from Language Models (ELMo), FastText, word2vec, latent semantic analysis (LSA), and char tri-gram feature models. Moreover, fine-tuning of the BERT is also performed. Three oversampling methods are investigated to handle the imbalance status of the Qprop dataset. SMOTE Edited Nearest Neighbors (ENN) presented the best results. The fine-tuning of BERT revealed that the BERT-320 sequence length is the best model. As a standalone model, the char tri-gram presented superior performance as compared to other features. The robust performance is observed against the combination of char tri-gram + BERT and char tri-gram + word2vec and they outperformed the two state-of-the-art baselines. In contrast to prior approaches, the addition of feature selection further improves the performance and achieved more than 97.60% recall, f1-score, and AUC on the dev and test part of the dataset. The findings of the present study can be used to organize news articles for various public news websites.

Categorization of tweets for damages: infrastructure and human damage assessment using fine-tuned BERT model

“…The BERT model was introduced by Devlin et al (2018) at Google Lab and it has proven its significance for a variety of text-mining tasks in several application domains ( Malik, Imran & Mamdouh, 2023 ). The benefits of BERT include faster development, automated feature generation, reduced data requirements, and improved performance.…”

“…TF-IDF is a statistical approach to evaluate the significance of a particular word in a large context of the document. This technique is commonly used in NLP and information retrieval (IR) tasks ( Malik, Imran & Mamdouh, 2023 ). It is a weighting technique and the weight of a word in a document is proportional to its frequency of occurrence whereas it is also inversely proportional to its frequency in all documents.…”

“…To capture the actual context of the language used to describe human and infrastructural damages in tweets, a state-of-the-art language model is required. Fine-tuning the BERT model has demonstrated robust performance in similar natural language processing (NLP) tasks ( Malik, Cheema & Ignatov, 2023 ; Malik, Imran & Mamdouh, 2023 ). Therefore, we are interested in utilizing the architecture of the BERT language model with fine-tuning.…”

Leveraging Prompt-Based Large Language Models: Predicting Pandemic Health Decisions and Outcomes Through Social Media Language

“…First, traditional NLP models and commonly used LLMs in CSS research often lack reasoning capabilities [116]. For instance, LLMs like BERT-based models, which are extensively used in HCI research that analyze large volumes of social media data [27,60], are typically fine-tuned for specific discrete downstream tasks (e.g., classification). While these pretrained language models have shown promise in performing discrete analyses, some emerging HCI research [116,117] demonstrate the additional value of prompting LLMs to perform multi-step reasoning for a more comprehensive analysis.…”

Fine-Tuning Transformer Models Using Transfer Learning for Multilingual Threatening Text Identification

“…Word2vec word embedding model has shown state-ofthe-art performance in many classification tasks related to the NLP domain [36][37][38]. There are two methods supported by word2vec to generate word embeddings; skip-gram and CBOW.…”