Abstract:Using capture-recapture analysis we estimate the effective size of the active Amazon Mechanical Turk (MTurk) population that a typical laboratory can access to be about 7,300 workers. We also estimate that the time taken for half of the workers to leave the MTurk pool and be replaced is about 7 months. Each laboratory has its own population pool which overlaps, often extensively, with the hundreds of other laboratories using MTurk. Our estimate is based on a sample of 114,460 completed sessions from 33,408 uni… Show more
“…The present studies also relied on participants recruited through Mturk. Recent work has noted that Mturk participants may have prior experience with canonical judgment and decision-making tasks (Stewart et al, 2015), and this experience may influence the strength of different experimental manipulations (Chandler, Paolacci, Peer, Mueller & Ratliff, 2015;Rand et al, 2014). However, we believe that task experience would -if anything -reduce the differences we observed in the present studies.…”
Recent research suggests that people discount or neglect expectations of reciprocity in trust dilemmas. We examine the underlying processes and boundary conditions of this effect, finding that expectations have stronger effects on trust when they are made accessible and when they are provided as objective probabilities (Study 1). Objective expectations have stronger effects when they are based on precise, rather than ambiguous, probabilities (Study 2). We also find that trust decisions differ from individual risk-taking decisions: people are more willing to trust, and expectations have stronger effects on trusting behavior (Study 2). These results show that the availability and ambiguity of expectations shape trust decisions, and that people differentially weight expectations in dilemmas of trust and individual risk-taking.
“…The present studies also relied on participants recruited through Mturk. Recent work has noted that Mturk participants may have prior experience with canonical judgment and decision-making tasks (Stewart et al, 2015), and this experience may influence the strength of different experimental manipulations (Chandler, Paolacci, Peer, Mueller & Ratliff, 2015;Rand et al, 2014). However, we believe that task experience would -if anything -reduce the differences we observed in the present studies.…”
Recent research suggests that people discount or neglect expectations of reciprocity in trust dilemmas. We examine the underlying processes and boundary conditions of this effect, finding that expectations have stronger effects on trust when they are made accessible and when they are provided as objective probabilities (Study 1). Objective expectations have stronger effects when they are based on precise, rather than ambiguous, probabilities (Study 2). We also find that trust decisions differ from individual risk-taking decisions: people are more willing to trust, and expectations have stronger effects on trusting behavior (Study 2). These results show that the availability and ambiguity of expectations shape trust decisions, and that people differentially weight expectations in dilemmas of trust and individual risk-taking.
“…Across seven studies conducted by Berinsky and colleagues (2012) with over 1,500 unique subjects, 30% of subjects had participated in more than one study (the mean number of studies completed per subject was 1.7). Similarly, Stewart et al (2015) find high rates of repeated participation within laboratories. Further, the majority of workers "follow" favorite requesters, and this practice is more common among the most prolific workers (Chandler et al, 2014).…”
Section: Introductionmentioning
confidence: 98%
“…In one study's sample, the average MTurk worker had completed a staggering 1,500 MTurk jobs, of which 300 were academic studies (Rand et al, 2014). Another recent analysis found that it takes about seven months for half of the pool of workers to be replaced (Stewart et al, 2015).…”
Section: Introductionmentioning
confidence: 99%
“…The issue of prior exposure may be further exacerbated by the fact that the pool of available workers is smaller than might be assumed. A recent analysis using capturerecapture analysis found that the average lab samples from a pool of about 7,300 MTurk workers (so a lab's reach in practice is much smaller than the population of 50,000 advertised by Amazon; Stewart et al, 2015). Further, a small number of extremely active workers within this pool (sometimes referred to as "professional" subjects) are responsible for large proportion of study responses (Chandler et al, 2014;Berinsky, Huber & Lenz, 2012).…”
Much research in cognitive psychology has focused on the tendency to conserve limited cognitive resources. The CRT is the predominant measure of such miserly information processing, and also predicts a number of frequently studied decision-making traits (such as belief bias and need for cognition). However, many subjects from common subject populations have already been exposed to the questions, which might add considerable noise to data. Moreover, the CRT has been shown to be confounded with numeracy. To increase the pool of available questions and to try to address numeracy confounds, we developed and tested the CRT-2. CRT-2 questions appear to rely less on numeracy than the original CRT but appear to measure closely related constructs in other respects. Crucially, substantially fewer subjects from Amazon’s Mechanical Turk have been previously exposed to CRT-2 questions. Though our primary purpose was investigating the CRT-2, we also found that belief bias questions appear suitable as an additional source of new items. Implications and remaining measurement challenges are discussed.
“…Moreover, low-quality responses and suspicious accesses (e.g., duplicate IP addresses, server farms) are blocked. To mitigate the 'super-worker problem' (i.e., a small group of workers that completes a disproportionately high share of tasks) and increase the pool of available MTurk workers (e.g., Chmielewski & Kucker, 2020;Stewart et al, 2015), we blocked the top-2% of workers from participating in our study (who complete ~34% of the tasks; Robinson et al, 2019).…”
In this registered report, we examined the effect of transgressions committed by leaders working at different group levels within an organization on employee outcomes. Based on social identity theorizing, we argued that organizational leader transgressions would affect organizational members’ experiences only at the organizational level, but that workgroup leader transgressions would impact organizational members’ experiences at both workgroup and organizational levels. To test these ideas, we developed a 2 (leader group affiliation: workgroup vs. organizational) × 2 (leader behaviour: normative vs. transgressive) between‐subjects experimental paradigm. As hypothesized, both workgroup and organizational leader transgressions resulted in decreased organizational identification and perceived organizational leader effectiveness. Contrary to our prediction, transgressions of both workgroup and organizational leaders were similarly detrimental to workers’ workgroup identification. However, as predicted, a transgressive workgroup leader had a greater negative impact on perceived workgroup leader effectiveness than a transgressive organizational leader. When outliers were excluded, a workgroup leader’s transgression was found to be more detrimental to work performance than an organizational leader’s transgression. Overall, this study demonstrates that the transgressions of lower‐level workgroup leaders can be as detrimental – and in some cases more detrimental – to workers than the transgressions of higher‐level organizational leaders.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.