Multilingual stance detection in social media political debates

Abstract

Stance Detection is the task of automatically determining whether the author of a text is in favor, against, or neutral towards a given target. In this paper we investigate the portability of tools performing this task across different languages, by analyzing the results achieved by a Stance Detection system (i.e. MultiTACOS) trained and tested in a multilingual setting. First of all, a set of resources on topics related to politics for English, French, Italian, Spanish and Catalan is provided which includes: novel corpora collected for the purpose of this study, and benchmark corpora exploited in Stance Detection tasks and evaluation exercises known in literature. We focus in particular on the novel corpora by describing their development and by comparing them with the benchmarks. Second, MultiTACOS is applied with different sets of features especially designed for Stance Detection, with a specific focus to exploring and combining both features based on the textual content of the tweet (e.g., style and affective load) and features based on contextual information that do not emerge directly from the text. Finally, for better highlighting the contribution of the features that most positively affect system performance in the multilingual setting, a features analysis is provided, together with a qualitative analysis of the misclassified tweets for each of the observed languages, devoted to reflect on the open challenges.

P-Stance: A Large Dataset for Stance Detection in Political Domain

“…The most common stance detection task on social media is target-specific stance detection (ALDayel and Magdy, 2021) which aims to identify the stance toward a set of figures or topics (Hasan and Ng, 2014;Mohammad et al, 2016a;Xu et al, 2016;Taulé et al, 2017;Swami et al, 2018;Zotova et al, 2020;Conforti et al, 2020b;Lai et al, 2020;Vamvas and Sennrich, 2020;Conforti et al, 2020a). Besides target-specific stance detection, multi-target stance detection Darwish et al, 2017;Li and Caragea, 2021a), and claimbased stance detection (Qazvinian et al, 2011;Derczynski et al, 2015;Ferreira and Vlachos, 2016;Bar-Haim et al, 2017;Rao and Pomerleau, 2017;Derczynski et al, 2017;Gorrell et al, 2019) are other popular trends of stance detection.…”

“…Even though stance detection has received a lot of attention, the annotated data are usually limited, which poses strong challenges to supervised models. Moreover, a limitation of existing datasets is that explicit mentions of targets and surface-level lexical cues that may expose the stance can be widely observed in the data (Mohammad et al, 2016a;Swami et al, 2018;Darwish et al, 2018;Conforti et al, 2020b;Lai et al, 2020), which means a model can detect the stance without extracting effective representations for the meanings of sentences (i.e., their lexical and compositional semantics). Another limitation of existing datasets, especially the datasets built on social media, is that the average length of tweets is short, which indicates that the data in these previous datasets are less informative and thus the stance can be detected more easily.…”

Towards Argument Mining for Social Good: A Survey

“…This approach is shown to perform well for classifying argument stance and detecting evidence, but not for predicting argument quality scores. Multilingual stance detection in political social media text (Vamvas and Sennrich, 2020) is also investigated in Lai et al (2020) using stylistic, structural, affective and contextual features from text and analysing the scenarios in which each of these features is effective.…”

Multilingual Argument Mining: Datasets and Analysis

“…Research on stance detection in a multilingual setting is rather recent. Zotova et al (2020) explore stance detection in Twitter for Catalan and Spanish; Lai et al (2020) do this for political debates in social media in these two languages as well as French and Italian; Vamvas and Sennrich (2020) analyze the stance of comments in the context of 1 https://www.research.ibm.com/ haifa/dept/vst/debating_data.shtml# MultilingualArgumentMining the Switzerland election in German, French and Italian. Stance detection is reminiscent of the Natural Language Inference (NLI) problem, where one is given two sentences, and the objective is to determine whether one entails the other, contradicts it or is neutral.…”