The Effects of Respondent and Question Characteristics on Respondent Answering Behaviors in Telephone Interviews

Abstract

In a standardized telephone interview, respondents ideally are able to provide an answer that easily fits the response task. Deviations from this ideal question answering behavior are behavioral manifestations of breakdowns in the cognitive response process and partially reveal mechanisms underlying measurement error, but little is known about what question characteristics or types of respondents are associated with what types of deviations. Evaluations of question problems tend to look at one question characteristic at a time; yet questions are comprised of multiple characteristics, some of which are easier to experimentally manipulate (e.g., presence of a definition) than others (e.g., attitude versus behavior). All of these characteristics can affect how respondents answer questions. Using a landline telephone interview, we use cross-classified random effects logistic regression models to simultaneously evaluate the effects of multiple question and respondent characteristics on six different respondent behaviors. We find that most of the variability in these respondent answering behaviors is associated with the questions rather than the respondents themselves. Question characteristics that affect the comprehension and mapping stages of the cognitive response process are consistently associated digitalcommons.unl.edu

The effect of emphasis in telephone survey questions on survey measurement quality

“…We controlled for the number of words in the question stem (Mean: WLT2 = 17.3, CS = 26.3). Longer questions will take longer to administer (e.g., Olson & Smyth, 2015), have more opportunities for the interviewer to misread or for the respondent to interrupt (e.g., Olson et al, 2019), and more opportunities for the interviewer to vary their pitch.…”

Constructing Behaviour Profiles for Answer Behaviour Across Surveys.

“…Considered from this approach, we note that we deliberately did not use a more classic method like cross-classified multilevel analysis (see for instance Olson & Smyth, 2015;Olson et al, 2019) that takes into account repeated measurements of individual respondents. The focus of our study was placed on visualizing summaries of undesirable answer behaviour and comparing subgroups who share the same characteristic.…”

“…Investigating data quality and modeling answer behaviour can be done by using item response models (Bolt & Johnson, 2009;De Jong, Steenkamp, Fox, & Baumgartner, 2008;Tijmstra, Bolsinova, & Jeon, 2018;Zickar, Gibby, & Robie, 2004), latent class models (Van Rosmalen et al, 2010), multitrait multimethod (MTMM) data (Andrews, 1984;Campbell & Fiske, 1959;Hox, 1995;Saris & Gallhofer, 2000Saris, Revilla, Krosnick, & Shaeffer, 2010;Saris, Satorra, & Coenders, 2004;Scherpenzeel, 1995;Scherpenzeel & Saris, 1997), multilevel models (Beretvas, 2011;Fielding & Goldstein, 2006;Gummer & Roßmann, 2014;Leckie, 2019;Olson & Smyth, 2015;Olson, Smyth, & Ganshert, 2019;Raudenbush & Bryk, 2002), or a combination of the two latter (Maas, Lensvelt-Mulders, & Hox, 2009). We need to note that we do not use a customary statistical method for the main part of this thesis.…”

“…Techniques that account for item and respondent characteristics in modelling answer behaviour are cross-classified multilevel models (see for instance Olson & Smyth, 2015;Olson et al, 2019) and mixed item response theory models (see for instance Tijmstra et al, 2018;Zickar et al, 2004). We have good reasons to provide an alternative method to model answer behaviour.…”

“…Thus, men and women may differ in various kinds of undesirable answer behaviour across surveys. Alwin & Krosnick 1991;Holbrook et al 2003;Kellogg 2007;Lavrakas 2010;Lavrakas et al 2010;Lynn & Kaminska 2013;Olson et al 2019;Schwarz et al 1991 Primary occupation 1. paid employment 2. family business or self-employed 3. job seeker 4. other: exempted from job seeking, work disability, unpaid/voluntary work 5. attends school or is studying 6. takes care of the housekeeping 7. retired Butler et al 1987;Lynn & Kaminska 2013;McClendon 1991;Schräpler 2004 Income 1. no income 2. 1 to 1000 EUR 3.…”