Social Media Analyses for Social Measurement

Abstract

Demonstrations that analyses of social media content can align with measurement from sample surveys have raised the question of whether survey research can be supplemented or even replaced with less costly and burdensome data mining of already-existing or "found" social media content. But just how trustworthy such measurement can be-say, to replace official statistics-is unknown. Survey researchers and data scientists approach key questions from starting assumptions and analytic traditions that differ on, for example, the need for representative samples drawn from frames that fully cover the population. New conversations between these scholarly communities are needed to understand the potential points of alignment and non-alignment. Across these approaches, there are major differences in (a) how participants (survey respondents and social media posters) understand the activity they are engaged in; (b) the nature of the data produced by survey responses and social media posts, and the inferences that are legitimate given the data; and (c) practical and ethical considerations surrounding the use of the data. Estimates are likely to align to differing degrees depending on the research topic and the populations under consideration, the particular features of the surveys and social media sites involved, and the analytic techniques for extracting opinions and experiences from social media. Traditional population coverage may not be required for social media content to effectively predict social phenomena to the extent that social media content distills or summarizes broader conversations that are also measured by surveys.

The empiricist’s challenge: Asking meaningful questions in political science in the age of big data

“…This makes the development of weighing procedures that would account for a stable skewness between the users of digital services and the general population unfeasible. Accordingly, public opinion scholars have warned against using digital trace data uncritically as proxies for other data sources (Murphy et al, 2014;Schober, Pasek, Guggenheim, Lampe, & Conrad, 2016). Add to these issues the very real challenges emerging from establishing a robust workflow with digital trace data in the social sciences (Freelon, 2015;King, 2011), and you find that the empiricist's promise seems much more elusive than originally thought.…”

New data sources for social indicators: the case study of contacting politicians by Twitter

“…Finally, the review by Schober et al (2016) on social media as data source shows that the diversity of studies makes it difficult to assess the potential of new data sources to measure socio-economic phenomena. According to their findings, studies will need to present results consistently aligned with the survey data when replicating the selected method over time.…”

Traditional versus Facebook-based surveys: Evaluation of biases in self-reported demographic and psychometric information

“…In the demographic research field, many studies regularly rely on surveys to tackle the interplay between psychological well-being and societal issues (Parr 2010;Moor and Komter 2012;Bernardi and Klärner 2014;Ho 2015;Teerawichitchainan, Knodel, and Pothisiri 2015;Morrison and Clark 2016). These are often administered to a scientifically constructed sample of the population, however, high-quality survey data require a substantial investment of time, effort, and resources (Wilson, Gosling, and Graham 2012;Schober et al 2016), especially in cases dealing with fast-evolving phenomena such as crisis response (Imran et al 2015) and deployment of resources during health emergencies (Vespignani 2009). In addition to providing rich observational data, social media platforms can also act as surveying tools, adding to existing traditional practices (Snelson 2016).…”