Measurement of Interviewer Workload within the Survey and an Exploration of Workload Effects on Interviewers’ Field Efforts and Performance

Abstract

Interviewer characteristics are usually assumed fixed over the fieldwork period. The number of sample units that require the interviewers’ attention, however, can vary strongly over the fieldwork period. Different workload levels produce different constraints on the time interviewers have available to contact, recruit and interview each target respondent, and may also induce different motivational effects on interviewers’ behavior as they perform their different tasks. In this article we show that fine-grained, time-varying operationalizations of project-specific workload can be useful to explain differences in interviewers’ field efforts and achieved response outcomes over the fieldwork period. We derive project-specific workload for each interviewer on each day of fieldwork in two rounds of the European Social Survey in Belgium from contact history and assignment paradata. Project-specific workload is measured as (1) the number of sample units which have been and remain assigned on any day t (assigned case workload), and (2) the number of sample units for which interviewer activity has started and not yet ceased on any day t (active case workload). Capturing temporal variation in interviewers’ workloads in a direct way, the time-varying operationalizations, are better predictors than are the interviewer-level operationalizations of typical (active or potential) workload that are derived from them, as well as the traditional total-count workload operationalization.

What can Interviewer-Collected Paradata Tell About Measurement Quality in Face-to-Face Surveys? Analyzing Response Styles in six Rounds of the European Social Survey

“…proposed in the literature (seeBlom, de Leeuw and Hox 2011;Blom 2012;West and Blom 2017;Wuyts and Loosveldt 2020), but we decided to measure the concept in the simplest possible way by counting the number of successful interviews conducted by the particular interviewer. A high number of active cases ascribed to an interviewer can result in lower data quality (e.g., Ackermann-Piek and Massing 2014; Wuyts and Loosveldt 2020), including cases caused by purposeful interviewers' actions(Menold and Kemper 2014).…”