Comparison of Self-Reported Telephone Interviewing and Web-Based Survey Responses: Findings From the Second Australian Young and Well National Survey

Abstract

BackgroundWeb-based self-report surveying has increased in popularity, as it can rapidly yield large samples at a low cost. Despite this increase in popularity, in the area of youth mental health, there is a distinct lack of research comparing the results of Web-based self-report surveys with the more traditional and widely accepted computer-assisted telephone interviewing (CATI).ObjectiveThe Second Australian Young and Well National Survey 2014 sought to compare differences in respondent response patterns using matched items on CATI versus a Web-based self-report survey. The aim of this study was to examine whether responses varied as a result of item sensitivity, that is, the item’s susceptibility to exaggeration on underreporting and to assess whether certain subgroups demonstrated this effect to a greater extent.MethodsA subsample of young people aged 16 to 25 years (N=101), recruited through the Second Australian Young and Well National Survey 2014, completed the identical items on two occasions: via CATI and via Web-based self-report survey. Respondents also rated perceived item sensitivity.ResultsWhen comparing CATI with the Web-based self-report survey, a Wilcoxon signed-rank analysis showed that respondents answered 14 of the 42 matched items in a significantly different way. Significant variation in responses (CATI vs Web-based) was more frequent if the item was also rated by the respondents as highly sensitive in nature. Specifically, 63% (5/8) of the high sensitivity items, 43% (3/7) of the neutral sensitivity items, and 0% (0/4) of the low sensitivity items were answered in a significantly different manner by respondents when comparing their matched CATI and Web-based question responses. The items that were perceived as highly sensitive by respondents and demonstrated response variability included the following: sexting activities, body image concerns, experience of diagnosis, and suicidal ideation. For high sensitivity items, a regression analysis showed respondents who were male (beta=−.19, P=.048) or who were not in employment, education, or training (NEET; beta=−.32, P=.001) were significantly more likely to provide different responses on matched items when responding in the CATI as compared with the Web-based self-report survey. The Web-based self-report survey, however, demonstrated some evidence of avidity and attrition bias.ConclusionsCompared with CATI, Web-based self-report surveys are highly cost-effective and had higher rates of self-disclosure on sensitive items, particularly for respondents who identify as male and NEET. A drawback to Web-based surveying methodologies, however, includes the limited control over avidity bias and the greater incidence of attrition bias. These findings have important implications for further development of survey methods in the area of health and well-being, especially when considering research topics (in this case diagnosis, suicidal ideation, sexting, and body image) and groups that are being recruited (young people, males, and NEET).

Using Participatory Design Methodologies to Co-Design and Culturally Adapt the Spanish Version of the Mental Health eClinic: Qualitative Study

“…The self-report assessment included 16 modules (Table 1 [64-89]) with smart skips built in so that it was tailored to each individual and took a minimum amount of time to complete (approximately 45 minutes).…”

Impact of physical appearance changes reported by adolescent and young adult cancer survivors: A qualitative analysis

“…This is consistent with evidence from general and cancer populations (Ata et al, ; Olsson et al, ; Vlachioti et al, ). However, it should be noted that men typically discuss body image concerns less in telephone interviews than other non‐verbal methods, such as self‐report surveys (Milton, Ellis, Davenport, Burns, & Hickie, ). Our data do suggest some male AYAs experience discomfort with physical appearance changes given their impact on physical activity routines and muscle mass, two factors associated with lower body satisfaction and perceived attractiveness (Frederick et al, ; Neumark‐Sztainer, Paxton, Hannan, Haines, & Story, ).…”

Symptoms of postpartum anxiety and depression among women in Canada: findings from a national cross-sectional survey

“…Further, unlike for the MES, where responses were collected by a human interviewer, responses to the SMH were largely collected through an online questionnaire. This allows for greater disclosure of symptoms of psychological distress than traditional interviewing methods, which involve a human interviewer, and may have contributed, to some extent, to the higher estimates observed (Milton et al 2017;Duffy et al 2005).…”