Web-Based Questionnaires and the Mode Effect

Abstract

Most methodological evaluations of web-based questionnaires have focused on the issues of sampling and response rates. Some have considered the issues of privacy and ethics. Relatively few have addressed the question of whether people provide different information depending on the mode of questionnaire delivery. This article contributes to this relatively overlooked aspect of the evaluation of web surveys. It presents initial findings from a survey that was designed to enable near-identical groups to respond to near-identical questionnaires delivered in different modes. Web-based questionnaires and paper-based questionnaires, used as part of a schoolbased study of young people's health-related behavior, are compared in terms of (a) completion rates and (b) data contents. Issues surrounding the quality of data and the reliability of webbased questionnaires are discussed. It is concluded that, on the basis of the quantitative data from this survey, there is little evidence of a mode effect linked to web-based questionnaires.

A comparison of stimulus ratings made online and in person: Gender and method effects

“…We will return to the anonymity manipulation below. Denscombe (2006) assigned groups of 15-year-old students from the same school to complete either a Web-based or a paper-and-pencil version of a survey of health-related behaviors. There was an unambiguously statistically different pattern of responding on only 1 of 23 questions.…”

Establishing the Preliminary Validity of Spiritual Eco-Maps with Native Americans

“…After IRB approval was received, the survey instrument was placed online and the URL link was emailed to individuals who agreed to participate in the study. Research suggests that response rates for paper and web-based surveys are comparable (Kaplowitz et al 2004) and the data produced is generally similar in content, particularly for quantitative items (Denscombe 2006). For open-ended, text-based questions, some evidence suggests that internet surveys may yield longer answers, although further research is needed to confirm this finding (Denscombe 2006).…”

Web-Based Surveys: Best Practices Based on the Research Literature

“…A series of validation studies in 1998 by Infratest Burke (as reported in Comley, 2002) showed results between the two methods were amazingly close with correlations of results .84 to .95 on all key measures. Denscombe (2006) found that for quantitative data there was little evidence of a mode effect, that is, comparable Web and paper surveys seemed to produce equivalent results.…”

“…Eysenbach provided a useful checklist for reporting results that addresses these issues. On a strict measure of completion, Denscombe (2006) reported that Web-based questionnaires do slightly better than paper questionnaires (97.1% vs. 81.8%).…”

“…General indications are that Web surveys are a reliable data collection method as measured against equivalent paper-based versions (Denscombe, 2006). The benefits that stem from Web efficiencies, however, do not provide sufficient grounds to always prefer Web-based surveys over paper-based surveys.…”