Internet Versus Mailed Questionnaires: A Randomized Comparison

Abstract

Background The use of Internet-based questionnaires for collection of data to evaluate patient education and other interventions has increased in recent years. Many self-report instruments have been validated using paper-and-pencil versions, but we cannot assume that the psychometric properties of an Internet-based version will be identical.Objectives To look at similarities and differences between the Internet versions and the paper-and-pencil versions of 16 existing self-report instruments useful in evaluation of patient interventions.Methods Participants were recruited via the Internet and volunteered to participate (N=397), after which they were randomly assigned to fill out questionnaires online or via mailed paper-and-pencil versions. The self-report instruments measured were overall health, health distress, practice mental stress management, Health Assessment Questionnaire (HAQ) disability, illness intrusiveness, activity limitations, visual numeric for pain, visual numeric for shortness of breath, visual numeric for fatigue, self-efficacy for managing disease, aerobic exercise, stretching and strengthening exercise, visits to MD, hospitalizations, hospital days, and emergency room visits. Means, ranges, and confidence intervals are given for each instrument within each type of questionnaire. The results from the two questionnaires were compared using both parametric and non-parametric tests. Reliability tests were given for multi-item instruments. A separate sample (N=30) filled out identical questionnaires over the Internet within a few days and correlations were used to assess test-retest reliability.Results Out of 16 instruments, none showed significant differences when the appropriate tests were used. Construct reliability was similar within each type of questionnaire, and Internet test-retest reliability was high. Internet questionnaires required less follow-up to achieve a slightly (non-significant) higher completion rate compared to mailed questionnaires.Conclusions Among a convenience sample recruited via the Internet, results from those randomly assigned to Internet participation were at least as good as, if not better than, among those assigned mailed questionnaires, with less recruitment effort required. The instruments administered via the Internet appear to be reliable, and to be answered similarly to the way they are answered when they are administered via traditional mailed paper questionnaires.

An online programme to reduce depression in patients with multiple sclerosis: a randomised controlled trial

“…Self-report questionnaires obtained online are similar to written questionnaires in terms of reliability and validity, 21 as shown for depression scales including the Beck Depression Inventory (BDI). 22 The primary endpoint of the study was severity of depressive symptoms as measured by the BDI.…”

Exploring relations among mindfulness facets and various meditation practices: Do they work in different ways?

“…The first is the relatively small sample size recruited on the Internet, which was probably more heterogeneous and may have resulted in a selection bias producing the underrepresentation of specific groups of people or meditation types, even though some studies confirm the reliability of data obtained from this source (Ritter, Lorig, Laurent, & Matthews, 2004). Second, indicators of meditation practice were reported through a recall method.…”

Long-Term Effects of an Internet-Based Treatment for Posttraumatic Stress

“…Ritter, Lorig, Laurent, and Matthews (2004) compared the psychometric properties of Internet-based versus mailed paper-and-pencil questionnaires. They found that questionnaires administered through the Internet were reliable and answered as often as the paper-and-pencil questionnaire.…”