Comparing Data Characteristics and Results of an Online Factorial Survey between a Population-Based and a Crowdsource-Recruited Sample

Weinberg, Jill D.; Freese, Jeremy; McElhattan, David

doi:10.15195/v1.a19

Cited by 472 publications

(389 citation statements)

References 23 publications

Supporting

Mentioning

345

Contrasting

Unclassified

Order By: Relevance

“…In all of our studies, we recruited participants from the subject pool of Amazon's Mechanical Turk (MTurk), an online crowdsourcing service with large volumes of small web-based tasks offered to anonymous online workers for monetary compensation. MTurk allows behavioral experiments to be run comparatively quickly and inexpensively, provides access to a broad cross-section of the population, and has repeatedly been shown to have the capacity to produce highly valid data (60)(61)(62)(63)(64)(65). The research was approved by the Institutional Review Board of the University of Arizona, and participants voluntarily agreed to take part after reading a disclosure form for research participation.…”

Section: Methodsmentioning

confidence: 99%

Power decreases trust in social exchange

Schilke

Reimann

Cook

2015

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

How does lacking vs. possessing power in a social exchange affect people's trust in their exchange partner? An answer to this question has broad implications for a number of exchange settings in which dependence plays an important role. Here, we report on a series of experiments in which we manipulated participants' power position in terms of structural dependence and observed their trust perceptions and behaviors. Over a variety of different experimental paradigms and measures, we find that more powerful actors place less trust in others than less powerful actors do. Our results contradict predictions by rational actor models, which assume that low-power individuals are able to anticipate that a more powerful exchange partner will place little value on the relationship with them, thus tends to behave opportunistically, and consequently cannot be trusted. Conversely, our results support predictions by motivated cognition theory, which posits that lowpower individuals want their exchange partner to be trustworthy and then act according to that desire. Mediation analyses show that, consistent with the motivated cognition account, having low power increases individuals' hope and, in turn, their perceptions of their exchange partners' benevolence, which ultimately leads them to trust. trust | power | social exchange | dependence | hope

show abstract

Section: Methodsmentioning

confidence: 99%

Power decreases trust in social exchange

Schilke

Reimann

Cook

2015

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

show abstract

“…MTurk has become popular among social scientists, particularly for conducting survey experiments in the fields of psychology, political science, sociology, and health, among others (Campbell and Gaddis forthcoming;Dowling and Miller 2016;Horne et al 2015). Researchers have praised MTurk for its relatively low cost and quick turnaround for data and have offered cautious optimism regarding generalizability (Horton, Rand, and Zeckhauser 2011;Weinberg, Freese, and McElhattan 2014). Moreover, on a number of dimensions, MTurk represents a superior alternative to using undergraduate students, a ubiquitous sample in experimental psychology (Sears 1986).…”

Section: Using Amazon's Mechanical Turkmentioning

confidence: 99%

“…Workers on MTurk also lean towards more liberal attitudes and opinions (Berinsky et al 2012). There is some evidence that these demographic differences account for minimal differences in effect sizes between MTurk and other Internet survey platforms that claim representative samples (Weinberg et al 2014). Moreover, careful checks of moderating demographic variables that are not representative of the United States in MTurk samples and/or weighting may alleviate concerns regarding external validity (Mullinix et al 2016;Weinberg et al 2014).…”

Section: Internal and External Validity With Mturkmentioning

confidence: 99%

How Black Are Lakisha and Jamal? Racial Perceptions from Names Used in Correspondence Audit Studies

Gaddis¹

2017

SocScience

213

View full text Add to dashboard Cite

Abstract:Online correspondence audit studies have emerged as the primary method to examine racial discrimination. Although audits use distinctive names to signal race, few studies scientifically examine data regarding the perception of race from names. Different names treated as black or white may be perceived in heterogeneous ways. I conduct a survey experiment that asks respondents to identify the race they associate with a series of names. I alter the first names given to each respondent and inclusion of last names. Names more commonly given by highly educated black mothers (e.g., Jalen and Nia) are less likely to be perceived as black than names given by less educated black mothers (e.g., DaShawn and Tanisha). The results suggest that a large body of social science evidence on racial discrimination operates under a misguided assumption that all black names are alike, and the findings from correspondence audits are likely sensitive to name selection.

show abstract

“…Berinsky, Huber, and Lenz 2012;Goodman, Cryder, and Cheema 2012;Horton, Rand, and Zeckhauser 2011;Krupnikov and Levine 2014;Paolacci, Chandler, and Ipeirotis 2010;Weinberg, Freese, and McElhattan 2014). 3 While these studies are impressive and telling, each includes only a small number of comparisons (e.g., three experiments) on a limited set of issues (e.g., three or four) and topics (e.g., question wording, framing) with few types of samples (e.g., three) at different points in time (e.g., data were collected on distinct samples far apart in time).…”

mentioning

confidence: 99%

The Generalizability of Survey Experiments

Mullinix¹,

Leeper²,

Druckman³

et al. 2015

J Exp Polit Sci

Self Cite

970

603

View full text Add to dashboard Cite

Survey experiments have become a central methodology across the social sciences. Researchers can combine experiments’ causal power with the generalizability of population-based samples. Yet, due to the expense of population-based samples, much research relies on convenience samples (e.g. students, online opt-in samples). The emergence of affordable, but non-representative online samples has reinvigorated debates about the external validity of experiments. We conduct two studies of how experimental treatment effects obtained from convenience samples compare to effects produced by population samples. In Study 1, we compare effect estimates from four different types of convenience samples and a population-based sample. In Study 2, we analyze treatment effects obtained from 20 experiments implemented on a population-based sample and Amazon's Mechanical Turk (MTurk). The results reveal considerable similarity between many treatment effects obtained from convenience and nationally representative population-based samples. While the results thus bolster confidence in the utility of convenience samples, we conclude with guidance for the use of a multitude of samples for advancing scientific knowledge.

show abstract

Comparing Data Characteristics and Results of an Online Factorial Survey between a Population-Based and a Crowdsource-Recruited Sample

Cited by 472 publications

References 23 publications

Power decreases trust in social exchange

Power decreases trust in social exchange

How Black Are Lakisha and Jamal? Racial Perceptions from Names Used in Correspondence Audit Studies

The Generalizability of Survey Experiments

Contact Info

Product

Resources

About