Can Amazon's Mechanical Turk be used to recruit participants for internet intervention trials? A pilot study involving a randomized controlled trial of a brief online intervention for hazardous alcohol use

Abstract

ObjectivesTo determine whether Amazon's Mechanical Turk (MTurk) might be a viable means of recruiting participants for online intervention research. This was accomplished by conducting a randomized controlled trial of a previously validated intervention with participants recruited through MTurk.MethodsParticipants were recruited to complete an online survey about their alcohol use through the MTurk platform. Those who met eligibility criterion for age and problem drinking were invited to complete a 3-month follow-up. Those who agreed were randomized to receive access to an online brief intervention for drinking or were assigned to a no intervention control group (i.e., thanked and told that they would be re-contacted in 3 months).ResultsA total of 423 participants were recruited, of which 85% were followed-up at 3-months. All participants were recruited in 3.2 h. Only 1/3 of participants asked to access the online brief intervention did so. Of the 4 outcome variables (number of drinks in a typical week, highest number on one occasion, number of consequences, AUDIT consumption subscale), one displayed a significant difference between conditions. Participants in the intervention group reported a greater reduction between on the AUDIT consumption subscale between baseline and 3-month follow-up compared to those in the no intervention control group (p = 0.004).ConclusionsDespite the current pilot showing only limited evidence of impact of the intervention among participants recruited through MTurk, there is potential for conducting trials employing this population (particularly if methods are employed to make sure that participants receive the intervention). This potential is important as it could allow for the rapid conduct of multiple trials during the development stages of online interventions.

Pilot randomized controlled trial of an online intervention for problem gamblers

“…This study is the fifth of a series of pilot RCTs employing this crowdsourcing platform to recruit participants for intervention research conducted by this group. Of the other four trials (all targeting unhealthy alcohol use), 2 of the 4 were able to demonstrate some small impact of the intervention under study ( Bertholet, Godinhno, & Cunningham, 2019 ; Cunningham et al, 2017a , Cunningham et al, 2017b ; Cunningham, Godinho, and Bertholet, under review ). However, all the trials, including the current one, suffered from difficulties with getting participants to engage with the intervention.…”

“…The present pilot RCT employed a crowdsourcing website as a quick and inexpensive method of recruiting participants for an RCT evaluating the impact of the online gambling intervention developed by Hodgins and colleagues ( Hodgins et al, 2013 ) and compared it to a no intervention control. While there are limitations associated with employing a sample recruited from Mechanical Turk (the crowdsourcing website) for this purpose ( Cunningham et al, 2017a , Cunningham et al, 2017b ), the results have value given the dearth of research in this area (although there are other trials with published protocols ongoing, e.g., ( Merkouris et al, 2017 )). It was hypothesized that the gambling intervention would lead to improved gambling outcomes at a 6-week and 6-month follow-up compared to participants in the no intervention control group.…”

Using Mechanical Turk to recruit participants for internet intervention research: experience from recruitment for four trials targeting hazardous alcohol consumption

“…It was possible to recruit large numbers of participants who drank in a hazardous fashion. In addition, many were interested in completing another survey (for a $10 payment) and, at least for the first trial, the follow-up rate was good (85%) [ 17 ]. However, MTurk does not contain an inexhaustible supply of participants who drink in a hazardous fashion [ 6 ].…”

Mechanical Turk data collection in addiction research: utility, concerns and best practices

“…These sampling methods must be carefully considered to recruit ideal participants (e.g. recruiting participants based on drinking rate, rather than Alcohol Use Disorders Identification Test (AUDIT) scores, for an intervention targeting drinking rate itself [27]). Individuals may also be recruited to complete extensive screening batteries to allow for the assignment of researcher-defined qualifications.…”