(Mis)alignment Between Stance Expressed in Social Media Data and Public Opinion Surveys

Abstract

Stance detection, which aims to determine whether an individual is for or against a target concept, promises to uncover public opinion from large streams of social media data. Yet even human annotation of social media content does not always capture "stance" as measured by public opinion polls. We demonstrate this by directly comparing an individual's selfreported stance to the stance inferred from their social media data. Leveraging a longitudinal public opinion survey with respondent Twitter handles, we conducted this comparison for 1,129 individuals across four salient targets. We find that recall is high for both "Pro" and "Anti" stance classifications but precision is variable in a number of cases. We identify three factors leading to the disconnect between text and author stance: temporal inconsistencies, differences in constructs, and measurement errors from both survey respondents and annotators. By presenting a framework for assessing the limitations of stance detection models, this work provides important insight into what stance detection truly measures.

Gender stereotypes may not influence the choice of female leaders: Experimental evidence from a crisis framed as social or economic during the COVID‐19 pandemic

“…However, our findings suggest that these COVID-19-related media posts might not have necessarily reflected or changed laypeople's opinions (see also Joseph et al, 2021). Our participants did not seem to perceive a significant difference in leadership style and change potential based on candidate gender nor did they generally prefer female over male leaders for high-status political positions in the COVID-19 crisis.…”

From Posts to Pavement, or Vice Versa? The Dynamic Interplay between Online Activism and Offline Confrontations

“…Our analysis seeks to understand how online resonance and legitimacy correlate with offline confrontations, measured by confrontational event frequency and property damage. Distinct from the general "public opinion" as measured by surveys (Horowitz et al 2023;Joseph et al 2021), online resonance is immediately visible to the public, thereby potentially influencing the movement's trajectory. This study uses Facebook data to gauge online resonance and to expand the theory into a computational framework, allowing the assessment of how online resonance is related to offline confrontation.…”