The effect of administration mode onCAHPSsurvey response rates and results: A comparison of mail and web‐based approaches

Abstract

Objective The objective of this study was to compare response rates, respondents' characteristics, and substantive results for CAHPS surveys administered using web and mail protocols. Data Sources Patients who had one or more primary care visits in the preceding 6 months. Study Design/Data Collection Methods Patients for whom primary care practices had email addresses were randomized to one of four survey administration protocols: web via a portal invitation; web via an email invitation; combination of web and mail; and mail only. Another sample of patients without known email addresses was surveyed by mail. Samples of nonrespondents to the Internet and mail protocols were surveyed by telephone. Principal Findings Response rates to surveys administered using the Internet protocols were lower than for the surveys administered by mail (20 percent vs over 40 percent). However, characteristics of respondents and survey answers were very similar across protocols. Respondents without email addresses were older, less educated, and more likely to be male than those with email addresses, and there were a few differences in their responses. There was little evidence of nonresponse bias in either the mail or web protocols. Conclusion In this well‐educated patient population, web protocols had lower response rates, but substantive results very similar to those from mail protocols.

Patient-reported experiences with general practitioners: a randomised study of mail and web-based approaches following a national survey

“…The results are consistent with a number of previous studies reporting that mail surveys achieve higher response rates than electronic and web-based approaches. [2][3][4][5][6][7][8][9][10][11] A recent Consumer Assessment of Healthcare Providers and Systems survey produced similar results when comparing protocols based on web responses via an email invitation and mail. 7 The mail protocol yielded more than twice the response rate of the web approach.…”

“…The effects of background characteristics reported in the literature are inconsistent. [2][3][4][5][6][7][8][9][10] The results from two previous randomised studies showed similar background characteristics for respondents in different randomised groups. 2 3 However, the respondents in those surveys were all contacted by mail.…”

“…Open access indicate that web mode surveys have lower response rates than other modes. [2][3][4][5][6][7][8][9][10][11] In the national patient experience survey among patients visiting GPs in 2014 in Norway, only 18% of respondents answered electronically. 4 However, the potential advantages of lower costs and shorter data-collection periods are important arguments for performing further research into web-based surveys.…”

“…The literature on the effects of background characteristics on the responses to different data collection methods are inconsistent. [2][3][4][5][6][7][8][9][10] Non-response bias has been studied in four patient populations in Norway through follow-up telephone interviews with nonrespondents, [12][13][14][15] including non-respondents in a survey on patient experiences with GPs. 15 The results have shown minor differences between the postal respondents from the national surveys and the postal non-respondents who have provided answers through follow-up interviews.…”

Internet-based surveys: relevance, methodological considerations and troubleshooting strategies

“…Examination of different methods of using web portals for surveying consumers about their experience with medical care revealed that response rates were higher for mail questionnaires, compared with web-based protocols. 6 A study on physicians 7 found that initial mailing of questionnaire followed by web surveying non-responders increased response rates and enhanced the representativeness of the sample. For surveys with shorter time frame, the reverse method, namely initial web survey followed by mailing the questionnaire to non-responders was recommended, though key outcome variables did not differ between these data collection methods.…”