Online panels in social science research: Expanding sampling methods beyond Mechanical Turk

Chandler, Jesse; Rosenzweig, Cheskie; Moss, Aaron J.; Robinson, Jonathan; Litman, Leib

doi:10.3758/s13428-019-01273-7

Cited by 455 publications

(374 citation statements)

References 53 publications

Supporting

Mentioning

364

Contrasting

Order By: Relevance

“…. These findings are in line with previous research showing that people with previous exposure to the CRT score higher than those without previous exposure [14]. Indeed, previous exposure to the CRT was positively correlated with performance (r = .31, p < .001).…”

Section: Resultssupporting

confidence: 92%

Tapped Out or Barely Tapped? Recommendations for How to Harness the Vast and Largely Unused Potential of the Mechanical Turk Participant Pool

Robinson¹,

Rosenzweig²,

Moss³

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

Mechanical Turk (MTurk) is a common source of research participants within the academic community. Despite MTurk's utility and benefits over traditional subject pools some researchers have questioned whether it is sustainable. Specifically, some have asked whether MTurk workers are too familiar with manipulations and measures common in the social sciences, the result of many researchers relying on the same small participant pool.Here, we show that concerns about non-naivete on MTurk are due less to the MTurk platform itself and more to the way researchers use the platform. Specifically, we find that there are at least 250,000 MTurk workers worldwide and that a large majority of US workers are new to the platform each year and therefore relatively inexperienced as research participants. We describe how inexperienced workers are excluded from studies, in part, because of the worker reputation qualifications researchers commonly use. Then, we propose and evaluate an alternative approach to sampling on MTurk that allows researchers to access inexperienced participants without sacrificing data quality. We recommend that in some cases researchers should limit the number of highly experienced workers allowed in their study by excluding these workers or by stratifying sample recruitment based on worker experience levels. We discuss the trade-offs of different sampling practices on MTurk and describe how the above sampling strategies can help researchers harness the vast and largely untapped potential of the Mechanical Turk participant pool.

show abstract

Section: Resultssupporting

confidence: 92%

Tapped Out or Barely Tapped? Recommendations for How to Harness the Vast and Largely Unused Potential of the Mechanical Turk Participant Pool

Robinson¹,

Rosenzweig²,

Moss³

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Similar to other online data collection platforms like Amazon Mechanical Turk (MTurk), CloudResearch’s Prime Panels allows for researchers to target and collect large, diverse samples of participants. Prime Panels participants have been shown to produce similar rates of passing attention checks and similar effect sizes as other online and in-lab samples, while being more representative of the U.S. population ( Chandler et al, 2019 ). Prime Panels also has a larger proportion of older adults, with over 23% of participants over the age of 60 relative to 3.3% of MTurk participants meeting the same criterion ( Chandler et al, 2019 ; Huff & Tingley, 2015 ).…”

Section: Studymentioning

confidence: 90%

Younger and Older Adults’ Mood and Expectations Regarding Aging During COVID-19

Whatley

Siegel

Schwartz

et al. 2020

Gerontology and Geriatric Medicine

View full text Add to dashboard Cite

The 2019 novel coronavirus disease (COVID-19) has broadly impacted our daily lives. Here, we used a longitudinal approach to investigate older adults’ mood and expectations regarding aging before and during the global pandemic (Study 1). We also examined age differences in mood, expectations regarding aging, COVID-19 attitudes, and loneliness using a cross-sectional approach (Study 2). In Study 1, older adults completed a mood and expectations regarding aging survey up to 2 years prior to the pandemic and again in April, 2020 (during the pandemic). Participants also completed surveys regarding COVID-19 attitudes and loneliness. In Study 2, a United States sample of younger and older adults completed these surveys during the pandemic. Older adults’ mood and expectations regarding aging remained fairly constant, and younger adults showed lower mood and expectations regarding aging than did older adults, despite older adults showing greater concern about COVID-19. Overall, we find that some older adults seem to be resilient with respect to their mood and expectations regarding aging. These findings reveal important preliminary implications for how older adults may be impacted as a result of lifestyle changes necessary for well-being and the well-being of society.

show abstract

“…The current study is based on an online research sample. Study participants came from Amazon Mechanical Turk (MTurk), a crowdsourcing website that is widely used to obtain high-quality data rapidly and inexpensively [ 35 , 36 , 37 , 38 ]. This study utilizes data from 30 survey questions that were distributed on Qualtrics, assessing (1) food store access, (2) dietary habits, (3) perceived weight status, (4) perceived health status, and (5) demographics.…”

Section: Methodsmentioning

confidence: 99%

Racial Differences in Perceived Food Swamp and Food Desert Exposure and Disparities in Self-Reported Dietary Habits

Stowers

Jiang

Atoloye

et al. 2020

IJERPH

View full text Add to dashboard Cite

Both food swamps and food deserts have been associated with racial, ethnic, and socioeconomic disparities in obesity rates. Little is known about how the distribution of food deserts and food swamps relate to disparities in self-reported dietary habits, and health status, particularly for historically marginalized groups. In a national U.S. sample of 4305 online survey participants (age 18+), multinomial logistic regression analyses were used to assess by race and ethnicity the likelihood of living in a food swamp or food desert area. Predicted probabilities of self-reported dietary habits, health status, and weight status were calculated using the fitted values from ordinal or multinomial logistic regression models adjusted for relevant covariates. Results showed that non-Hispanic, Black participants (N = 954) were most likely to report living in a food swamp. In the full and White subsamples (N = 2912), the perception of residing in a food swamp/desert was associated with less-healthful self-reported dietary habits overall. For non-Hispanic Blacks, regression results also showed that residents of perceived food swamp areas (OR = 0.66, p < 0.01, 95% CI (0.51, 0.86)) had a lower diet quality than those not lving in a food swamp/food desert area. Black communities in particular may be at risk for environment-linked diet-related health inequities. These findings suggest that an individual’s perceptions of food swamp and food desert exposure may be related to diet habits among adults.

show abstract

Online panels in social science research: Expanding sampling methods beyond Mechanical Turk

Cited by 455 publications

References 53 publications

Tapped Out or Barely Tapped? Recommendations for How to Harness the Vast and Largely Unused Potential of the Mechanical Turk Participant Pool

Tapped Out or Barely Tapped? Recommendations for How to Harness the Vast and Largely Unused Potential of the Mechanical Turk Participant Pool

Younger and Older Adults’ Mood and Expectations Regarding Aging During COVID-19

Racial Differences in Perceived Food Swamp and Food Desert Exposure and Disparities in Self-Reported Dietary Habits

Contact Info

Product

Resources

About