Identifying Multiple Submissions in Internet Research: Preserving Data Integrity

Abstract

Internet-based sexuality research with hidden populations has become increasingly popular. Respondent anonymity may encourage participation and lower social desirability, but associated disinhibition may promote multiple submissions, especially when incentives are offered. The goal of this study was to identify the usefulness of different variables for detecting multiple submissions from repeat responders and to explore incentive effects. The data included 1,900 submissions from a three-session Internet intervention with a pretest and three post-test questionnaires. Participants were men who have sex with men and incentives were offered to rural participants for completing each questionnaire. The final number of submissions included 1,273 "unique", 132 first submissions by "repeat responders" and 495 additional submissions by the "repeat responders" (N = 1,900). Four categories of repeat responders were identified: "infrequent" (2-5 submissions), "persistent" (6-10 submissions), "very persistent" (11-30 submissions), and "hackers" (more than 30 submissions). Internet Provider (IP) addresses, user names, and passwords were the most useful for identifying "infrequent" repeat responders. "Hackers" often varied their IP address and identifying information to prevent easy identification, but investigating the data for small variations in IP, using reverse telephone look up, and patterns across usernames and passwords were helpful. Incentives appeared to play a role in stimulating multiple submissions, especially from the more sophisticated "hackers". Finally, the web is ever evolving and it will be necessary to have good programmers and staff who evolve as fast as "hackers". KeywordsInternet research; Multiple submissions; Repeat responders; Validity; Rural; MSM Internet-based research with high HIV risk and hidden populations, such as men who have sex with men (MSM), is rapidly expanding (Pequegnat et al. 2006). The increase in Internet research has led a number of authors to call for examination of methodological issues related to internal and external validity (Kraut et al. 2004;Pequegnat et al. 2006). The anonymity of the Internet provides advantages and disadvantages in terms of internal validity (Kraut et al. 2004). Respondent anonymity may reduce pressure to respond in socially desirable ways, but may lead to unintended outcomes. Catania (1999) suggests that lowered presentation bias may result in more honest responses to sensitive questions. However, anonymity in online studies may lower self-regulation, increasing the possibility of developing multiple identities (Joinson 1998). If significant numbers of participants in online studies change their identity and enroll multiple times, the integrity of data will be severely compromised (Birnbaum 2004;Mustanski 2001;Pequegnat et al. 2006; Reips 2002a, b).Early in the Internet research revolution, it was suggested that the prevalence of repeat responders (the number of people who enroll multiple times) and the frequency of multiple submissions (the numbe...

Sexual health promotion for young people delivered via digital media: a scoping review

“…The potential anonymity of the online environment raises concerns about invalid data owing to, for example, multiple enrolment or deliberate falsification. 233,246 Multiple enrolment has been reported as occurring in an online sexual health survey as well as in online trials of digital interventions for sexual health. 246 Multiple enrolment can be minimised by collecting several different personal identifiers (e.g.…”

“…233,246 Multiple enrolment has been reported as occurring in an online sexual health survey as well as in online trials of digital interventions for sexual health. 246 Multiple enrolment can be minimised by collecting several different personal identifiers (e.g. name, e-mail address, postal address and telephone number), and by requesting information (e.g.…”

Patterns of synthetic cannabinoid use in Australia

“…Offering reimbursement or prizes provides motivation for multiple responding or vote stacking [20] and also involves taking identifying information from participants. Instead, we emphasised the anonymous nature of participation in our study, which did not collect Internet Protocol (IP) addresses or email addresses, which could potentially be used to identify individuals.…”

“…Instead, we emphasised the anonymous nature of participation in our study, which did not collect Internet Protocol (IP) addresses or email addresses, which could potentially be used to identify individuals. The disadvantage of not collecting IP addresses is that we cannot identify respondents who may have taken the survey more than once from the same computer [20]; however, the lack of financial incentive and the 15 min time commitment reduced the motivation for respondents to deliberately respond more than once. Although we did not collect IP addresses within the dataset, we did use Google Analytics to conduct traffic analysis on the URL used to promote the survey, kronicstudy.net.…”

“…Users reported buying a median of 3 g (IQR 3-6) and paying a median of AU$60 (IQR 37-90). The median price paid per gram was AU$18 (IQR [13][14][15][16][17][18][19][20][21][22][23].…”