Web Versus Other Survey Modes: An Updated and Extended Meta-Analysis Comparing Response Rates

Abstract

<jats:title>Abstract</jats:title> <jats:p>Do web surveys still yield lower response rates compared with other survey modes? To answer this question, we replicated and extended a meta-analysis done in 2008 which found that, based on 45 experimental comparisons, web surveys had an 11 percentage points lower response rate compared with other survey modes. Fundamental changes in internet accessibility and use since the publication of the original meta-analysis would suggest that people’s propensity to participate in web surveys has changed considerably in the meantime. However, in our replication and extension study, which comprised 114 experimental comparisons between web and other survey modes, we found almost no change: web surveys still yielded lower response rates than other modes (a difference of 12 percentage points in response rates). Furthermore, we found that prenotifications, the sample recruitment strategy, the survey’s solicitation mode, the type of target population, the number of contact attempts, and the country in which the survey was conducted moderated the magnitude of the response rate differences. These findings have substantial implications for web survey methodology and operations.</jats:p>

Going web or staying paper? The use of web-surveys among older people

“…Previous studies show that web surveys in general produce approximately 10-11% lower response rate than other survey modes, such as paper and telephone (7,28). A recent meta-analyses, including over 100 experiments, con rm these results and show a 12 % response rate difference between web-surveys and other modes (19). The exception is among students, were the results are more mixed.…”

Recruitment of women veterans into suicide prevention research: Improving response rates with enhanced recruitment materials and multiple survey modalities

“…Many factors influence response rates, including characteristics of the recruitment base (e.g., demographics, population type), sampling and recruitment methods (e.g., probability sampling from a known sampling frame with direct invitation of a purposefully selected group vs. non‐probability sampling of an easily accessible group), number of contact attempts, survey completion incentives (e.g., monetary compensation), survey sponsorship (e.g. governmental, academic, commercial), as well as survey topic, length, and modality (Daikeler, BosnJak, & Manfreda 2019; Fan & Yan, 2010; Guo, Kopec, Cibere, Li, & Goldsmith, 2016).…”

“…Although response rate trends have been examined across populations and survey modalities (Daikeler et al, 2019), there is a dearth of research exploring nonresponse bias and its impact in survey research with veterans. This is true both among the veteran population as a whole, as well as in specific veteran subgroups for which there is a critical need to improve understanding of health and health care utilization.…”

Going web or staying paper? The use of web-surveys among older people

“…A recent meta-analyses, including over 100 experiments, con rm these results and show a 12 % response rate difference between web-surveys and other modes (19). The exception is among students, were the results are more mixed.…”

“…The expectation is that previous problems with under-coverage in web-surveys will decrease when internet use is becoming more spread across all societal groups (18). So far, response-rates have been found to be consistently lower in web-surveys compared with other survey modes (19) and web-surveys have been found to have lower survey representativeness compared with other single mode surveys (5). The exception is among younger age groups, were a web-survey is likely to generate similar response rates as a paper-survey (3,13).…”