Transitions from Telephone Surveys to Self-Administered and Mixed-Mode Surveys: AAPOR Task Force Report

Abstract

Telephone surveys have been a ubiquitous method of collecting survey data, but the environment for telephone surveys is changing. Many surveys are transitioning from telephone to self-administration or combinations of modes for both recruitment and survey administration. Survey organizations are conducting these transitions from telephone to mixed modes with only limited guidance from existing empirical literature and best practices. This article summarizes findings by an AAPOR Task Force on how these transitions have occurred for surveys and research organizations in general. We find that transitions from a telephone to a self-administered or mixed-mode survey are motivated by a desire to control costs, to maintain or improve data quality, or both. The most common mode to recruit respondents when transitioning is mail, but recent mixed-mode studies use only web or mail and web together as survey administration modes. Although early studies found that telephone response rates met or exceeded response rates to the self-administered or mixed modes, after about 2013, response rates to the self-administered or mixed modes tended to exceed those for the telephone mode, largely because of a decline in the telephone mode response rates. Transitioning offers opportunities related to improved frame coverage and geographic targeting, delivery of incentives, visual design of an instrument, and cost savings, but challenges exist related to selecting a respondent within a household, length of a questionnaire, differences across modes in use of computerization to facilitate skip patterns and other questionnaire design features, and lack of an interviewer for respondent motivation and clarification. Other challenges related to surveying youth, conducting surveys in multiple languages, collecting nonsurvey data such as biomeasures or consent to link to administrative data, and estimation with multiple modes are also prominent.

Transitioning the Healthy Chicago Survey From a Telephone Mode to Self-administered by Mail Mode

“…Higher response rates can be achieved at a lower cost by conducting surveys in a self-administered mode by mail contact sampling addresses (a proxy for households) from an ABS frame 2…”

La Falla de las Encuestas en las Elecciones Argentinas de 2019. Un Análisis en Perspectiva Comparada Internacional (The Failure of the Polls in the 2019 Argentine Elections. An Analysis in International Comparative Perspective)

“…En EEUU y otros países muchas organizaciones han emprendido una transición a modos autoadministrados y/o mixtos de recolección de datos, combinando Internet, el teléfono, el correo postal y la entrevista presencial (Dillman, Smyth y Christian, 2014). La American Association for Public Opinion Research (AAPOR) elaboró informes que analizan las estrategias utilizadas en esa transición y ofrecen lineamientos para adoptar los nuevos modos (Olson et al, 2019;Lavrakas et al, 2017). Los objetivos que impulsan esta conversión son mejorar las tasas de respuesta, la cobertura de la población y la exactitud de las estimaciones, así como reducir costos.…”

Jewish Community Studies in the Twenty-First Century

“…Collecting data by telephone has become more difficult and more costly as the population has transitioned from landline telephone service toward primarily or exclusively cell phone service. As stated previously, new technologies to identify and/or block incoming calls, and respondents having less "free time" to answer surveys has fueled the need to change the approach to collecting population-level data (Olson et al 2019).…”

“…Studies have demonstrated that a well-designed ABS study attains response rates higher than those using RDD methods (Olson et al 2019;Montaquila et al 2013), especially when one considers that the non-list ABS frame is absent those who are most likely to respond positively to the named sponsor. In addition, knowing the location for ABS nonrespondents allows for adjustments in the weighting process that reduces a major source of potential bias in telephone-based data collection.…”