Selection Bias in Web Surveys

Abstract

SummaryAt first sight, web surveys seem to be an interesting and attractive means of data collection. They provide simple, cheap, and fast access to a large group of potential respondents. However, web surveys are not without methodological problems. Specific groups in the populations are under-represented because they have less access to Internet. Furthermore, recruitment of respondents is often based on self-selection. Both under-coverage and self-selection may lead to biased estimates. This paper describes these methodological problems. It also explores the effect of various correction techniques (adjustment weighting and use of reference surveys). This all leads to the question whether properly design web surveys can be used for data collection. The paper attempts to answer this question. It concludes that under-coverage problems may solve itself in the future, but that self-selection leads to unreliable survey outcomes.

Web based health surveys: Using a Two Step Heckman model to examine their potential for population health analysis

“…This is in contrast to the theory of probability sampling where each individual must have an equal chance of being included in the survey for the sample to generate unbiased results (Kish, 1965;Bethlehem, 2010;Sanchez-Fernandez et al, 2012). Self-selection into a survey implies that the principles of probability sampling are not followed (Berk, 1983;Lee and Marsh, 2000;Bethlehem, 2010). When this situation arises it is referred to as sample selection bias (Rubin, 1976;Berk, 1983).…”

“…The majority of web surveys allow participants to self-select into the sample (Schonlau et al, 2004;Bethlehem, 2010). Respondents are those people who happen to visit the website and then decide to participate in the survey (Bethlehem, 2010).…”

“…Table 1 presents an overview of the advantages and disadvantages associated with Internet surveys. The main drawback commonly cited with web-based surveys is potentially low levels of access to, and use of Internet services across the general population (Bethlehem, 2010). However, Internet coverage and access is improving rapidly across all demographics (Bethlehem, 2010;Strabac & Aalberg, 2011;ONS, 2015).…”

“…The main drawback commonly cited with web-based surveys is potentially low levels of access to, and use of Internet services across the general population (Bethlehem, 2010). However, Internet coverage and access is improving rapidly across all demographics (Bethlehem, 2010;Strabac & Aalberg, 2011;ONS, 2015). In England, a recent survey on internet usage by the Office of National Statistics (ONS, 2015) found that between January and March 2015, 86.2% of the English population had used the Internet in the last 3 months (recent users), while 2% (1.1 million) had last used the Internet more than 3 months ago and 11% (5.9 million) had never used it.…”

Are Canadians more willing to provide chest-compression-only cardiopulmonary resuscitation (CPR)?—a nation-wide public survey

“…Furthermore, recruitment of participants tends to be based on self-selection. Both under-coverage and self-selection may lead to biased estimates, 56 although the direction and magnitude are difficult to estimate given the increasingly wide availability of the Internet. Despite the potential bias, we feel that this was the most feasible approach for achieving a larger sample of members of the Canadian general public.…”