The importance of assessing clinical phenomena in Mechanical Turk research.

Abstract

Amazon.com's Mechanical Turk (MTurk) website provides a data collection platform with quick and inexpensive access to diverse samples. Numerous reports have lauded MTurk as capturing high-quality data with an epidemiological sample that is more representative of the U.S. population than traditional in-person convenience samples (e.g., undergraduate subject pools). This benefit, in combination with the ease and low-cost of data collection, has led to a remarkable increase in studies using MTurk to investigate phenomena across a wide range of psychological disciplines. Multiple reports have now examined the demographic characteristics of MTurk samples. One key gap remains, however, in that relatively little is known about individual differences in clinical symptoms among MTurk participants. This paper discusses the importance of assessing clinical phenomena in MTurk samples and supports its assertions through an empirical investigation of a large sample (N = 1,098) of MTurk participants. Results revealed that MTurk participants endorse clinical symptoms to a substantially greater degree than traditional nonclinical samples. This distinction was most striking for depression and social anxiety symptoms, which were endorsed at levels comparable with individuals with clinically diagnosed mood and anxiety symptoms. Participants' symptoms of physiological anxiety, hoarding, and eating pathology fell within the subclinical range. Overall, the number of individuals exceeding validated clinical cutoffs was between 3 and 19 times the estimated 12-month prevalence rates. Based on the current findings, it is argued that MTurk participants differ from the general population in meaningful ways, and researchers should consider this when referring to this sample as truly representative. (PsycINFO Database Record

National norms for the expanded version of the inventory of depression and anxiety symptoms (IDAS‐II)

“…A growing body of literature has examined the prevalence of internalizing symptoms and disorders in MTurk samples, yielding conflicting results. Whereas some findings have suggested that MTurk workers report higher rates of depression and anxiety symptoms than do community sample participants (Arditte et al., ), other studies have suggested either comparable (Shapiro, Chandler, & Mueller, ) or lower (Veilleux, Salomaa, Shaver, Zielinsky, & Pollert, ) levels of such symptoms in MTurk samples.…”

Examination of interoception along the suicidality continuum

“…Finally, although our sample provided a broader perspective on this association than samples of undergraduate students or patients with a particular disorder, our findings may not be generalizable to more specific populations, particularly high‐risk groups such as psychiatric inpatients. Moreover, although this study was intended to capture a representative community sample, recent research indicates that MTurk participants may have higher rates of social anxiety and depression than the general population (Arditte et al., ; Shapiro et al., )—and therefore may also have higher than average rates of suicidality. Although we were unable to examine these constructs in the present sample, it is possible that the self‐selective and voluntary nature of MTurk participation altered our findings—given that this study was presented as a study examining “diet, exercise, personality, and mental health.” As such, this study may not be truly representative of the general community, although it is certainly more representative than studies of undergraduate students or psychiatric patients.…”

Positive Emotion Infusions: Can Savoring Increase Help‐Seeking Intentions among People with Depression?

“…All participants were recruited through Amazon's Mechanical Turk, an online method of accessing a large participant pool by posting tasks online and providing monetary compensation for their completion (MTurk; Buhrmester, Kwang, & Gosling, 2011). Recent findings suggest that MTurk has a higher percentage of individuals with clinical depressive symptomatology compared to the general population (approximately 32%; Arditte, C ß ek, Shaw, & Timpano, 2016). Of those at or above the clinical cutoff score of 14 on the depression inventory used in the study, 53 reported mild, 50 reported moderate, and 32 reported severe symptomatology (M depression = 23.51, SD depression = 8.78).…”