Are Mechanical Turk worker samples representative of health status and health behaviors in the U.S.?

Abstract

IntroductionAmazon’s Mechanical Turk (MTurk) is frequently used to administer health-related surveys and experiments at a low cost, but little is known about its representativeness with regards to health status and behaviors.MethodsA cross-sectional survey comprised of questions from the nationally-representative 2014 Behavioral Risk Factor Surveillance System (BRFSS) and 2014 National Health and Nutrition Examination Survey (NHANES) was administered to 591 MTurk workers and 393 masters in 2016. Health status (asthma, depression, BMI, and general health), health behaviors (influenza vaccination, health insurance, smoking, and physical activity), and demographic characteristics of the two MTurk populations (workers and masters) were compared to each other and, using Poisson regression, to a nationally-representative BRFSS and NHANES samples.ResultsWorkers and master demographics were similar. MTurk users were more likely to be aged under 50 years compared to the national sample (86% vs. 55%) and more likely to complete a college degree than the national sample (50% vs. 26%). Adjusting for covariates, MTurk users were less likely to be vaccinated for influenza, to smoke, to have asthma, to self-report being in excellent or very good health, to exercise, and have health insurance but over twice as likely to screen positive for depression relative to a national sample. Results were fairly consistent among different age groups.ConclusionsMTurk workers are not a generalizable population with regards to health status and behaviors; deviations did not follow a trend. Appropriate health-related uses for MTurk and ways to improve upon the generalizability of MTurk health studies are proposed.

Relative importance of attribute preferences for radial vs. femoral arterial access: A crowdsourcing study of healthy online‐recruited volunteers

“…While participants recruited from MTURK tend to be younger than most individuals requiring coronary angiography or PCI, our study did not find any major differences in attribute preferences between those <50‐years old and those >50‐years old. Our participants were also healthier than the typical patient requiring a coronary artery procedure.…”

“…22 Furthermore, recruitment through MTURK can be inexpensive compared with other methods such as organizing focus groups or paying individuals to participate in lab studies. 22 While participants recruited from MTURK tend to be younger than most individuals requiring coronary angiography or PCI, 23 our study did not find any major differences in attribute preferences between those <50-years old and those >50-years old. Our participants were also healthier than the typical patient requiring a coronary artery procedure.…”

Hope and health‐related quality of life among chronically ill uninsured/underinsured adults

“…One study suggested that the health of Turkers is not representative of the population health of the United States (Walters et al, ). Though that study (i.e., Walters et al, ) mostly consisted of unidimensional indicators of physical health (i.e., influenza vaccination, smoking, physical activity, and asthma) and one unidimensional indicator of mental health (i.e., depression). Second, the present study relied solely on self‐report and cross‐sectional data.…”

“…Furthermore, MTurk can be used to access populations that are not traditionally represented in research because of the difficulty in accessing them (Smith, Sabat, Martinez, Weaver, & Xu, ). For example, Turkers (i.e., MTurk participants) are less likely to have health insurance than a nationally representative sample (Walters, Christakis, & Wright, ). The inclusion criteria for the present study were: (a) being uninsured or underinsured, (b) having at least one chronic health condition, (c) residing in the United States, (d) being 18 years of age or older, (e) being able to read in English, and (f) having access to the internet.…”

Associations of Weight Bias with Disordered Eating Among Latino and White Men

“…Similarly, we did not include male‐oriented measures of body image (e.g., drive for muscularity) that would provide a better understanding of how weight bias influences eating/weight‐related psychopathology in men. An additional limitation is the use of a self‐selected sample from Mechanical Turk; although they are more diverse than college samples, these self‐selected samples still tend to be younger and more educated than national samples . It is also unclear whether individuals with obesity in this study had internalized the weight‐biased attitudes they endorsed.…”