Paper- or Web-Based Questionnaire Invitations as a Method for Data Collection: Cross-Sectional Comparative Study of Differences in Response Rate, Completeness of Data, and Financial Cost

Abstract

BackgroundPaper questionnaires have traditionally been the first choice for data collection in research. However, declining response rates over the past decade have increased the risk of selection bias in cross-sectional studies. The growing use of the Internet offers new ways of collecting data, but trials using Web-based questionnaires have so far seen mixed results. A secure, online digital mailbox (e-Boks) linked to a civil registration number became mandatory for all Danish citizens in 2014 (exemption granted only in extraordinary cases). Approximately 89% of the Danish population have a digital mailbox, which is used for correspondence with public authorities.ObjectiveWe aimed to compare response rates, completeness of data, and financial costs for different invitation methods: traditional surface mail and digital mail.MethodsWe designed a cross-sectional comparative study. An invitation to participate in a survey on help-seeking behavior in out-of-hours care was sent to two groups of randomly selected citizens from age groups 30-39 and 50-59 years and parents to those aged 0-4 years using either traditional surface mail (paper group) or digital mail sent to a secure online mailbox (digital group). Costs per respondent were measured by adding up all costs for handling, dispatch, printing, and work salary and then dividing the total figure by the number of respondents. Data completeness was assessed by comparing the number of missing values between the two methods. Socioeconomic variables (age, gender, family income, education duration, immigrant status, and job status) were compared both between respondents and nonrespondents and within these groups to evaluate the degree of selection bias.ResultsA total 3600 citizens were invited in each group; 1303 (36.29%) responded to the digital invitation and 1653 (45.99%) to the paper invitation (difference 9.66%, 95% CI 7.40-11.92). The costs were €1.51 per respondent for the digital group and €15.67 for paper group respondents. Paper questionnaires generally had more missing values; this was significant in five of 17 variables (P<.05). Substantial differences were found in the socioeconomic variables between respondents and nonrespondents, whereas only minor differences were seen within the groups of respondents and nonrespondents.ConclusionsAlthough we found lower response rates for Web-based invitations, this solution was more cost-effective (by a factor of 10) and had slightly lower numbers of missing values than questionnaires sent with paper invitations. Analyses of socioeconomic variables showed almost no difference between nonrespondents in both groups, which could imply that the lower response rate in the digital group does not necessarily increase the level of selection bias. Invitations to questionnaire studies via digital mail may be an excellent option for collecting research data in the future. This study may serve as the foundational pillar of digital data collection in health care research in Scandinavia and other countries considering implementing s...

Factors Influencing Malaysian Consumers’ Intention to Use Quick Response (QR) Mobile Payment

“…Not only this technique is to ensure that the amount of responses collected sufficiently meets the requirement of this study, but also to widen the chances for those without online facilities to be the respondents. In this case, filling out a paper questionnaire is a practical and easy method because the respondent only needs a pen and time to participate (Ebert et al 2018). Prior to the distribution of the survey questionnaire, the questionnaire was first pre-tested with five experts from the academia in the field of marketing and technology.…”

Employees' experiences of education and knowledge in intellectual disability practice

“…In educational research, online survey has become a popular data collection method. However, the response rates of online surveys may be lower than those of paper-based questionnaires (Ebert, Huibers, Christensen, & Christensen, 2018). One reason for low response rate observed in this study could be that the email addresses were invalid, inactive, or seldom used.…”

Perspectives of online surveys in dermatology

“…A study on surgeons from the Orthopedic Trauma Association showed lower response rate in the Internet arm (45%) than the mailed arm (58%) . Ebert and colleagues sent 7200 invitations (3600 digital and 3600 paper) to Danish citizens to take part in a survey about help‐seeking behaviour in some diseases, and the response rates were 36.3% and 46% for the digital and paper arms, respectively . So et al .…”

“…23 Ebert and colleagues sent 7200 invitations (3600 digital and 3600 paper) to Danish citizens to take part in a survey about help-seeking behaviour in some diseases, and the response rates were 36.3% and 46% for the digital and paper arms, respectively. 19 So et al 35 conducted a web-based trial on physicians from the Japan Primary Care Association and the response rate hit a low of 6.3%.…”