Human Research and Data Collection via the Internet

Abstract

Advantages and disadvantages of Web and lab research are reviewed. Via the World Wide Web, one can efficiently recruit large, heterogeneous samples quickly, recruit specialized samples (people with rare characteristics), and standardize procedures, making studies easy to replicate. Alternative programming techniques (procedures for data collection) are compared, including client-side as opposed to server-side programming. Web studies have methodological problems; for example, higher rates of drop out and of repeated participation. Web studies must be thoroughly analyzed and tested before launching on-line. Many studies compared data obtained in Web versus lab. These two methods usually reach the same conclusions; however, there are significant differences between college students tested in the lab and people recruited and tested via the Internet. Reasons that Web researchers are enthusiastic about the potential of the new methods are discussed.

A randomized trial found online questionnaires supplemented by postal reminders generated a cost-effective and generalizable sample but don’t forget the reminders

“…Appearance and layout also matters: advertising on Google and Facebook have reportedly outperformed that of online news media, suggesting that the presentation of an online approach can affect its reception [30,31]. Future research should investigate how best to pilot e-mail and online surveys [32], specifically whether usability engineering and user experience methods may be able to increase the appeal of online and e-mail surveys through tailoring them to match the motivations and values of the target population.…”

Gender and Sexual Orientation Differences in Cognition Across Adulthood: Age Is Kinder to Women than to Men Regardless of Sexual Orientation

“…To obtain sufficient numbers of participants to allow analyses by age, gender, and sexual orientation, data were gathered via the Internet (for reviews of online experimentation, see Birnbaum, 2004;Reips, 2002). This has recently become popular because of the time, energy, and resources that can be saved by remotely testing substantial numbers of participants over a short period.…”

Identifying Multiple Submissions in Internet Research: Preserving Data Integrity

“…Early in the Internet research revolution, it was suggested that the prevalence of repeat responders (the number of people who enroll multiple times) and the frequency of multiple submissions (the number of times a single individual enrolls) were "rare" and "easy to detect" (Birnbaum 2000(Birnbaum , 2004Musch and Reips 2000; Reips 2002a, b). In these early studies, incentives were seldom used to encourage enrollment or increase retention (Birnbaum 2004).…”

“…However, anonymity in online studies may lower self-regulation, increasing the possibility of developing multiple identities (Joinson 1998). If significant numbers of participants in online studies change their identity and enroll multiple times, the integrity of data will be severely compromised (Birnbaum 2004;Mustanski 2001;Pequegnat et al 2006; Reips 2002a, b).Early in the Internet research revolution, it was suggested that the prevalence of repeat responders (the number of people who enroll multiple times) and the frequency of multiple submissions (the number of times a single individual enrolls) were "rare" and "easy to detect" (Birnbaum 2000(Birnbaum , 2004Musch and Reips 2000; Reips 2002a, b). In these early studies, incentives were seldom used to encourage enrollment or increase retention (Birnbaum 2004).…”

“…While anonymity may be useful, it may also inhibit researchers' ability to identify participants who attempt to join a study multiple times. Early studies suggested that the frequency of multiple submissions in Internet-based sexuality research was rare (Birnbaum 2004; Reips 2002a, b) and probably related to the use of incentives to encourage participation.The use of incentives in this study appears to have affected both the number of repeat responders and the number of submissions by an individual repeat responder and, like the Konstan et al(2005) study; we found a high frequency of multiple submissions. The men designated as eligible for compensation were six times more likely to be repeat responders than the 'unpaid' group.…”