Online volunteer laboratories for human subjects research

Abstract

Once a fixture of research in the social and behavioral sciences, volunteer subjects are now only rarely used in human subjects research. Yet volunteers are a potentially valuable resource, especially for research conducted online. We argue that online volunteer laboratories are able to produce high-quality data comparable to that from other online pools. The scalability of volunteer labs means that they can produce large volumes of high-quality data for multiple researchers, while imposing little or no financial burden. Using a range of original tests, we show that volunteer and paid respondents have different motivations for participating in research, but have similar descriptive compositions. Furthermore, volunteer samples are able to replicate classic and contemporary social science findings, and produce high levels of overall response quality comparable to paid subjects. Our results suggest that online volunteer labs represent a potentially significant untapped source of human subjects data.

Late Adulthood, COVID-19-Related Stress Perceptions, Meaning in Life, and Forgiveness as Predictors of Mental Health During the COVID-19 Pandemic

“…Limitations regarding sample, measurement, and design and analysis of this study should be considered. First, despite the Harvard DLABSS panel being a diverse one (Strange et al, 2019), the respondents to this study were overwhelmingly White, highly educated, and middle-upper income. This is a considerable limitation, given that so much of the COVID-19 pandemic's impact was felt disproportionately by people of color and those with less education and income (McKnight-Eily et al, 2021).…”

What Do Workers Want From (Private) Government? Experimental Evidence from the United States

“…respondents. Strange et al (2019) show that these online volunteer respondents have demographic characteristics comparable to other online panels. Additionally, they reproduce classic and contemporary social science findings and show response quality comparable to paid respondents.…”

Left Out But “In Control”? Culture Variations in Perceived Control When Excluded by a Close Other

“…To satisfy this criterion while accounting for potential data exclusion due to the attention checks, we recruited the following number of participants per study: (1) Study 1a: 175 Asians/Asian Americans, 285 European Americans; (2) Study 1b: 393 Asians/Asian Americans, 701 European Americans; (3) Study 2: 296 Japanese, 281 European Americans; and (4) Study 3: 239 Asians/Asian Americans, 850 European Americans. 2 We conducted Studies 1a, 1b, and 3 via online platforms hosted in the United States, using four different methods: (1) a volunteer laboratory called Digital Lab for the Social Sciences that is suggested to be equally or more representative than comparable services (e.g., Amazon's MTurk) and, importantly, to have respondents who are more naive about psychological methods (Strange et al, 2019); (2) a similar, yet separate, volunteer laboratory of general (i.e., mostly nonstudent) participants, hosted by a university the lead author was affiliated with; (3) two different pools of students participating for credit in their psychology classes, one at an elite private university in the Northeastern United States and one at a public university on the U.S. West Coast; and (4) LISTSERVS across the United States specifically for Asians (e.g., Asian American Association). Unless otherwise noted, participants were entered into drawings for $50 gift cards.…”