Paper, E-mail, or Both? Effects of Contact Mode on Participation in a Web Survey of Establishments

Abstract

Identifying strategies that maximize participation rates in population-based web surveys is of critical interest to survey researchers. While much of this interest has focused on surveys of persons and households, there is a growing interest in surveys of establishments. However, there is a lack of experimental evidence on strategies for optimizing participation rates in web surveys of establishments. To address this research gap, we conducted a contact mode experiment in which establishments selected to participate in a web survey were randomized to receive the survey invitation with login details and subsequent reminder using a fully crossed sequence of paper and e-mail contacts. We find that a paper invitation followed by a paper reminder achieves the highest response rate and smallest aggregate nonresponse bias across all-possible paper/e-mail contact sequences, but a close runner-up was the e-mail invitation and paper reminder sequence which achieved a similarly high response rate and low aggregate nonresponse bias at about half the per-respondent cost. Following up undeliverable e-mail invitations with supplementary paper contacts yielded further reductions in nonresponse bias and costs. Finally, for establishments without an available e-mail address, we show that enclosing an e-mail address request form with a prenotification letter is not effective from a response rate, nonresponse bias, and cost perspective.

Mental health and illness of medical students and newly graduated doctors during the pandemic of SARS-Cov-2/COVID-19

“…The present study has as main limitations the low response rates, which is expected in web surveys compared to other data collection modes [ 28 , 29 ], the reduction of the sample size in M2 in relation to M1, the impossibility of matching responses per person due to the mandatory anonymity, and the existence of questions asked only in M1 or in M2, due to the nexus relationship with the epidemiological moment.…”

The IAB Job Vacancy Survey: design and research potential

“…The second best solution in the tradeoff of costs, response rate and nonresponse bias is an email invitation followed by a paper reminder. As it has the lowest response rate and the highest level of bias, an email-only contact strategy should be avoided (Sakshaug et al 2019).…”

Surveying Across Borders: The Experiences of the German Emigration and Remigration Panel Study

“…The pushto-web design applied by GERPS follows a mixed-mode approach with differences between contact mode and survey response mode. This offers us the possibility to combine advantages of offline mail recruitment (Sakshaug et al 2019) with the benefits of an online survey (Cernat and Lynn 2017;Evans and Mathur 2018;Lee et al 2018).…”