Measuring Controversy in Social Networks Through NLP

Abstract

Nowadays controversial topics on social media are often linked to hate speeches, fake news propagation, and biased or misinformation spreading. Detecting controversy in online discussions is a challenging task, but essential to stop these unhealthy behaviours.In this work, we develop a general pipeline to quantify controversy on social media through content analysis, and we widely test it on Twitter.Our approach can be outlined in four phases: an initial graph building phase, a community identification phase through graph partitioning, an embedding phase, using language models, and a final controversy score computation phase. We obtain an index that quantifies the intuitive notion of controversy.To test that our method is general and not domain-, language-, geography-or size-dependent, we collect, clean and analyze 30 Twitter datasets about different topics, half controversial and half not, changing domains and magnitudes, in six different languages from all over the world.The results confirm that our pipeline can quantify correctly the notion of controversy, reaching a ROC AUC score of 0.996 over controversial and non-controversial scores distributions. It outperforms the state-of-the-art approaches, both in terms of accuracy and computational speed.

A SHAP-based controversy analysis through communities on Twitter

“…Our work is based on Twitter and focuses on tweets related to several topics, controversial or not. We perform our analysis on 30 different datasets provided in [4], retrieved using the Twitter API. 15 topics have been manually labeled controversial and 15 noncontroversial from multiple sources on mainstream media [4].…”

“…We perform our analysis on 30 different datasets provided in [4], retrieved using the Twitter API. 15 topics have been manually labeled controversial and 15 noncontroversial from multiple sources on mainstream media [4]. Non-controversial topics contain soft news such as entertainment or noticeable events with no controversy, while Controversial topics are mainly focused on political events (election, justice cases).…”

“…Tweets are cleaned up beforehand by replacing URLs and user tags with unique special tokens. From the data we got access to, some tweets might be missing in our datasets, depending on the topic, as tweets could have been deleted since the last time it was retrieved in [4]. The resulting dataset consists of 30 topics with their number of tweets ranging from 5 458 to 36 716, involving a number of users ranging from 3 696 to 161 612 per topic [24].…”

“…Based on those results, we can assume that in this controversial case, LIWC can help characterize a tendency in a community in relation to another. Looking for psychological state tendencies in communities 4 , figure 5 shows that top LIWC features impacting the prediction of rf liwc on test pelosi are from the categories "Tone", "function", "Period", "Exclam", "OtherP", "Cognition", "Affect", "Social", "Lifestyle", "Physical" and "Time orientation", presented in table 1. We notice that punctuation plays an important role ("Exclam", "OtherP", "period").…”

“…Recent studies have integrated textual information with structural data to enhance controversy classification tasks [3]. Notably, natural language processing (NLP) and deep learning techniques have been employed to enrich structural graph information and quantify controversy [4]. Understanding user behaviors in the context of impactful content has emerged as a significant challenge.…”