To follow or not to follow - Estimating political opinion from Twitter data using network analysis and machine learning

Abstract

Studying public opinion stands as a fundamental pursuit for both policymakers and researchers. While traditional surveys remain the primary method to investigate individual political opinions, the advent of social media data (SMD) offers novel prospects. However, the number of studies using SMD to extract individuals’ political opinions are limited and differ greatly in their methodological approaches and levels of success. Recent studies highlight the benefits of analyzing individuals’ network structure to estimate political opinions. Nevertheless, current methodologies exhibit limitations, including the use of simplistic linear models that disregard the complexity of relationships within the network, as well as a predominant focus on the United States. Addressing these issues, we employ a Variational Autoencoder (VAE) machine learning model to extract individual opinion estimates from SMD of N = 276 008 German Twitter (now: X) users, compare its performance to a state-of-the-art linear model and validate model estimates on self-reported opinion measures. Our findings suggest that the VAE captures the network structure of Twitter users more accurately, leading to higher accuracy of predicting following decisions as well as correlations with self-reported political ideology and voting intentions. Our study emphasizes the necessity of advanced analytical approaches, capable of capturing complex relationships in social media networks when studying public opinion, at least in non-US contexts. This research expands the understanding of utilizing SMD for public opinion analysis and underscores the potency of machine learning techniques in enhancing the predictive accuracy of SMD.