Positive Learning or Deviant Interviewing? Mechanisms of Experience on Interviewer Behavior

Abstract

Interviewer (mis)behavior has been shown to change with interviewers’ professional experience (general experience) and experience gained during the field period (survey experience). We extend this study by using both types of experiences to analyze interviewer effects on a core quality indicator: interview duration. To understand whether the effect of interviewer experience on duration is driven by increased efficiency or deviant behavior—both mechanisms of shorter interview durations—we additionally examine the triggering rate of filter questions to avoid burdensome follow-up questions and response differentiation over the field period. Using multilevel models and data from a large-scale survey on a special and difficult-to-interview population of refugees in Germany, we find that interview duration decreases with increasing survey experience, particularly among the generally inexperienced interviewers. However, this effect is not found for the triggering rate and response differentiation. The results are robust to different sample and model specifications. We conclude that the underlying mechanism driving interview duration is related to increasing efficiency, and not deviant behavior.

How to Catch a Falsifier

“…We acknowledge that the results are based on a single dataset and data collection could be subject to specific opportunities and motives for the interviewers to falsify ( Kosyakova et al 2021 ). Hence, while the results are encouraging, these methods could work out differently for other datasets.…”

“…We refer to this interviewer as “F1.” F1 accounted for 289 person interviews and 217 household interviews, which must be considered as complete falsifications. Further investigations carried out by the survey organization and the IAB (including various statistical methods, recontacting of respondents, and questioning of supervisors and interviewers) confirmed two additional falsifiers responsible for a total of 62 person and 47 household interviews ( DIW 2019 ; Kosyakova et al 2019 ). These interviewers did not fabricate all of their assigned interviews.…”

“…We utilize data from the IAB-BAMF-SOEP Survey of Refugees in Germany ( Brücker, Rother, and Schupp 2017 ), including verified falsifications (version SOEP.v33) ( Kosyakova et al 2019 ). 1 This is an annually conducted longitudinal household survey, launched in 2016.…”