Effects of Question Characteristics on Item Nonresponse in Telephone and Web Survey Modes

Abstract

Telephone surveys face more and more criticism because of decreasing coverage and increasing costs, and the risk of producing socially desirable answers. Consequently, survey administrators consider switching their surveys to the web mode, although the web mode is more susceptible to item nonresponse. Still, we do not know whether this is true for all question types. In this article, we analyze to what extent item nonresponse depends on question characteristics such as their form or difficulty in the telephone and the web mode. We use data from an experiment in which individuals randomly sampled from a population register are experimentally assigned to these two modes. Distinguishing effects on the frequency of don’t know responses, item refusals, and mid-scale responding, we find more don’t know responses and item refusals for the web mode generally, but no differences for mid-scale responding. However, this relationship depends on the characteristics of the question.

Use of interviewer-administered telephone surveys during infectious disease outbreaks, epidemics and pandemics: a scoping review

“…Adequate consideration for questions themselves is also the key to good IATS. For instance, characteristics of the questions, such as their form or difficulty, influence item non-response to some extent 57. The use of simple, easy words should be used, especially when interviewing or surveying the vulnerable.…”

Effects of Changing Modes on Item Nonresponse in Panel Surveys

“…However, an interviewer may also exert social pressure to give a substantively valid answer even if the respondent does not want or cannot answer a question (Chang and Krosnick 2010). INR often remains "hidden" in interviewer-based surveys as respondents, instead of not answering a question, may provide an answer that is socially accepted, for example a mid-scale response (Lipps and Monsch 2022;Sturgis et al 2014). The mode of interview also affects responses through variation in other characteristics, such as the pace of the interview, presentation (visual or auditive), and the timing of the interview (Christian et al 2007).…”

“…While Lipps and Monsch (2022) used data from the individual questionnaire of wave 1 of this experiment, we use data from the household questionnaire answered by the HRP in wave 1 and 2 in this article, because only for the household questionnaire did a modeswitch take place. Unlike Lipps and Monsch (2022), we are not able to distinguish question characteristics such as length or complexity of the questions, as the questions included in the household questionnaire were mostly factual questions.…”

“…The exception concerns the dummy variable indicating whether the HRP is older than 54: the tel-to-tel group is more likely to include these older respondents in wave 2 compared with the tel-to-web group. Like Lipps and Monsch (2022), to analyze the effects of respondent characteristics in the tel-to-tel and the tel-to-web conditions, we transform the data to a "cross-classified" structure (see Fielding and Goldstein 2006). Thus, we account for the crossed structure of the two levels respondents and questions: questions are clustered in respondents and respondents are clustered in questions.…”

Use of Interviewer-Administered Telephone Surveys during Infectious Disease Outbreaks, Epidemics, and Pandemics: A Scoping Review

“…Lastly, adequate consideration for questions themselves is also the key to good IATS. For instance, characteristics of the question influences item nonresponse since to some extent [29]. It should also be remembered that keeping surveys or interviews short (no longer than about 20-30 minutes) [6,30] can contribute to minimizing informants' fatigue [2].…”