Demographic Stability on Mechanical Turk Despite COVID-19

Abstract

a Numbers are percentages of unique participants within each month from the CloudResearch database. A description of this database and how well it captures the Mechanical Turk population appears in Robinson et al. [7] and Moss et al. [8]. To examine the percentage of studies completed by people within each demographic group each month, visit https://metrics.cloudresearch.com/ Trends in Cognitive Sciences

Trust in a COVID-19 vaccine in the U.S.: A social-ecological perspective

“…The protocols followed MTurk's best practices, which included ensuring participant confidentiality, protecting study integrity, generating unique completion codes, integrating attention-checks throughout the survey, repeating study-specific qualification questions, and removing disqualified participants ( Chandler and Shapiro, 2016 ; Strickland and Stoops, 2019 ; Young and Young, 2019 ). Moreover, despite COVID-19, the demographic characteristics of Mturk appear to be stable ( Moss et al, 2020 ). Eligibility included being age 18 or older, living in the United States, being able to speak and read English, having heard of the coronavirus or COVID-19, and providing written informed consent.…”

Changes in self-reported health, alcohol consumption, and sleep quality during the COVID-19 pandemic in the United States

“…Of course, the latter requires considerably more time and resources to employ, which was not feasible or even appropriate during a quickly-evolving pandemic. However, MTurk has the advantage over other data sources of demographic stability during the pandemic (Moss, Rosenzweig, and Robinson 2020).…”

Pipelines Bent, Pipelines Broken: Interdisciplinary Self-Reflection on the Impact of COVID-19 on Current and Future Research (Position Paper)

“…For crowd-sourced setups, there appears to be no noticeable difference so far in terms of the subject pool available. Demographic data showing who used the platform prior to the pandemic compares favorably with data from today [30,33]. Yet, we could also reasonably assume that the participant pool could change in the long term, with participants becoming less diverse [31].…”