A Comparison of Web and Telephone Responses From a National HIV and AIDS Survey

Abstract

BackgroundResponse differences to survey questions are known to exist for different modes of questionnaire completion. Previous research has shown that response differences by mode are larger for sensitive and complicated questions. However, it is unknown what effect completion mode may have on HIV and AIDS survey research, which addresses particularly sensitive and stigmatized health issues.ObjectivesWe seek to compare responses between self-selected Web and telephone respondents in terms of social desirability and item nonresponse in a national HIV and AIDS survey.MethodsA survey of 2085 people in Canada aged 18 years and older was conducted to explore public knowledge, attitudes, and behaviors around HIV and AIDS in May 2011. Participants were recruited using random-digit dialing and could select to be interviewed on the telephone or self-complete through the Internet. For this paper, 15 questions considered to be either sensitive, stigma-related, or less-sensitive in nature were assessed to estimate associations between responses and mode of completion. Multivariate regression analyses were conducted for questions with significant (P≤.05) bivariate differences in responses to adjust for sociodemographic factors. As survey mode was not randomly assigned, we created a propensity score variable and included it in our multivariate models to control for mode selection bias.ResultsA total of 81% of participants completed the questionnaire through the Internet, and 19% completed by telephone. Telephone respondents were older, reported less education, had lower incomes, and were more likely from the province of Quebec. Overall, 2 of 13 questions assessed for social desirability and 3 of 15 questions assessed for item nonresponse were significantly associated with choice of mode in the multivariate analysis. For social desirability, Web respondents were more likely than telephone respondents to report more than 1 sexual partner in the past year (fully adjusted odds ratio (OR)=3.65, 95% CI 1.80-7.42) and more likely to have donated to charity in the past year (OR=1.63, 95% CI 1.15-2.29). For item nonresponse, Web respondents were more likely than telephone respondents to have a missing or “don’t know” response when asked about: the disease they were most concerned about (OR=3.02, 95% CI 1.67-5.47); if they had ever been tested for HIV (OR=8.04, 95% CI 2.46-26.31); and when rating their level of comfort with shopping at grocery store if the owner was known to have HIV or AIDS (OR=3.11, 95% CI 1.47-6.63).ConclusionSociodemographic differences existed between Web and telephone respondents, but for 23 of 28 questions considered in our analysis, there were no significant differences in responses by mode. For surveys with very sensitive health content, such as HIV and AIDS, Web administration may be subject to less social desirability bias but may also have greater item nonresponse for certain questions.

Questionnaire Breakoff and Item Nonresponse in Web-Based Questionnaires: Multilevel Analysis of Person-Level and Item Design Factors in a Birth Cohort

“…Besides the design of the items, their content could also influence item nonresponse [26]. For instance, items asking about sensitive subjects could have higher nonresponse than items with nonsensitive content [34]. However, we did not perceive this behavior in our study.…”

Rural and Urban Differences in COVID‐19 Prevention Behaviors

“…For instance, some respondents could over‐report positive behaviors like mask‐wearing even if they typically do not wear a mask in public and under‐report negative behaviors like not washing their hands to appear to be following public health guidelines. Notably, however, research does suggest that social desirability bias for self‐reported health behaviors is typically lower in online surveys like ours 39 . Relatedly, in asking individuals to reflect on the past month, individuals could simply forget whether they engaged in each activity, potentially leading to some degree of error in each of our measures.…”

The Impact of Assessment Modality and Demographic Characteristics on Endorsement of Military Sexual Trauma

“…These findings do not seem to be driven entirely by privacy concerns; if these concerns were paramount, we would expect the greatest number of women to endorse MST on the DRRI. There is some evidence that interviews can produce higher estimates of certain behaviors because interviewers can clarify questions and motivate participants (e.g., Jones et al, 2016); this factor may explain why female participants endorsed MST most frequently on the study interview.…”