Using crowdsourced online experiments to study context-dependency of behavior

Abstract

We use Mechanical Turk's diverse participant pool to conduct online bargaining games in India and the US. First, we assess internal validity of crowdsourced experimentation through variation of stakes ($0, $1, $4, and $10) in the Ultimatum and Dictator Game. For cross-country equivalence we adjust the stakes following differences in purchasing power. Our marginal totals correspond closely to laboratory findings. Monetary incentives induce more selfish behavior but, in line with most laboratory findings, the particular size of a positive stake appears irrelevant. Second, by transporting a homogeneous decision situation into various living conditions crowdsourced experimentation permits identification of context effects on elicited behavior. We explore contextdependency using session-level variation in participants' geographical location, regional affluence, and local social capital. Across "virtual pools" behavior varies in the range of stake effects. We argue that quasi-experimental variation of the characteristics people bring to the experimental situation is the key potential of crowdsourced online designs.

On the Transportability of Laboratory Results

“…As a side effect, we cannot fully rule out the possibility that differences across samples and contexts may be partly due to differences in monetary incentives. Given the well-documented finding that specific sizes of positive stakes have negligible effects in interactive games of fairness, trust, and reciprocity (Carpenter et al 2005;Keuschnigg et al 2016), it is highly unlikely that stake differences drive our results. In fact, we find the lowest level of prosocial behavior in the setup providing the smallest stakes (Study 4)-a finding that runs counter the idea that prosociality decreases in stake sizes.…”

“…Critics may find fault at our heterogeneous stake levels pointing to the idea that observed prosociality may decrease in stake sizes. Prior evidence from laboratory (e.g., Camerer and Hogarth 1999;Carpenter, Verhoogen, and Burks 2005) and online studies (e.g., Amir et al 2012;Keuschnigg, Bader, and Bracher 2016), however, indicates that-although monetary stakes increase selfishness compared to unincentivized games-differences in positive stakes have negligible effects on laboratory results in fairness and cooperation research.…”

Bounded Solidarity in Cross-National Encounters: Individuals Share More with Others from Poor Countries but Trust Them Less

“…Each participant received a $1 show-up fee, and we incentivized decisions ($2 endowment in the DG, $1 for each player in the TG). Stake levels in this range have proven sufficient to minimize social-desirability effects on MTurk (Keuschnigg, Bader, and Bracher 2016). Participants received on average $1.86.…”

The effect of social and personal norms on stated preferences for multiple soil functions: evidence from Australia and Italy

“…While many approaches exist to measure social norms in incentivised laboratory and online experiments (e.g. Keuschnigg et al., 2016; Krupa & Weber, 2013) and field experiments (e.g. Baldassarri & Abascal, 2017; Winter & Zhang, 2018), they are less suitable in the context of our study.…”