A Biased Review of Biases in Twitter Studies on Political Collective Action

Abstract

In recent years researchers have gravitated to Twitter and other social media platforms as fertile ground for empirical analysis of social phenomena. Social media provides researchers access to trace data of interactions and discourse that once went unrecorded in the offline world. Researchers have sought to use these data to explain social phenomena both particular to social media and applicable to the broader social world. This paper offers a minireview of Twitter-based research on political crowd behavior. This literature offers insight into particular social phenomena on Twitter, but often fails to use standardized methods that permit interpretation beyond individual studies. Moreover, the literature fails to ground methodologies and results in social or political theory, divorcing empirical research from the theory needed to interpret it. Rather, investigations focus primarily on methodological innovations for social media analyses, but these too often fail to sufficiently demonstrate the validity of such methodologies. This minireview considers a small number of selected papers; we analyse their (often lack of) theoretical approaches, review their methodological innovations, and offer suggestions as to the relevance of their results for political scientists and sociologists.

Understanding Popularity, Reputation, and Social Influence in the Twitter Society

“…Our research has followed a theory‐driven approach, aiming at aspects of Twitter that are relevant for society as a whole and that have been inferred from both qualitative and computational models. This allowed us take a step further from descriptive and data‐driven analyses without theoretical context, framing our findings in a wider scientific perspective beyond the computational sciences (Cihon & Yasseri, ). We learned that popularity and reputation are not always motivating, that popularity is not as heterogeneous as was thought to be, and that popularity and reputation are both relevant when studying social influence.…”

Online reactions to the 2017 ‘Unite the right’ rally in Charlottesville: measuring polarization in Twitter networks using media followership

“…We collected Tweets with the hashtag #Charlottesville and the follower lists for 13 media organizations using Twitter's API and the Python package tweepy. Public data accessibility through Twitter's API has greatly facilitated research studies on Twitter data, but such data have important limitations [5,13], including potential biases due to Twitter's proprietary API sampling scheme [13]. For example, Morstatter et al [31] illustrated that the API can produce artifacts in topical tweet volume, potentially resulting in misleading changes in the number of tweets on a given topic over time.…”

“…It is common to analyze them individually as retweet (e.g., see [8,9]), follower (e.g., see [10]), mention (e.g., see [8]) networks, and others. An extensive literature is concerned with Twitter network data, and the myriad topics that have been studied using them include political protest and social movements [11][12][13][14][15][16][17], epidemiological surveillance and monitoring of health behaviors [18][19][20][21][22][23][24], contagion and online content propagation [25,26], identification of extremist groups [27], ideological polarization [8,28,29], and much more. Indeed, the combination of significance for public discourse, data accessibility, and amenability to network analysis is appealing.…”

Tampering with Twitter’s Sample API

“…Several studies discuss working, compositions and possible biases of data [47,48] and a "reverse-engineered" model has been developed for the Sample API, which indicates that the sampling is based on a millisecond time window and that the timestamp at which the Tweet arrived at Twitter's servers is coded into the Tweet's ID [42,43]. Although it has been shown that Twitter creates nonrepresentative samples with non-transparent and highly fluctuating sample rates of the overall Twitter activity [49], this has had no effect on its popularity amongst researchers [50]. It was suggested in the past that Sample API data can be used to estimate the quality of Streaming API data [51].…”