A Comparison Between Responses From a Propensity-Weighted Web Survey and an Identical RDD Survey

Abstract

The authors conducted a large-scale survey about health care twice, once as a web and once as a random digit dialing (RDD) phone survey. The web survey used a statistical technique, propensity scoring, to adjust for selection bias. Comparing the weighted responses from both surveys, there were no significant response differences in 8 of 37 questions. Web survey responses were significantly more likely to agree with RDD responses when the question asked about the respondent’s personal health (9 times more likely), was a factual question (9 times more likely), and only had two as opposed to multiple response categories (17 times more likely). For three questions, significant differences turned insignificant when adjacent categories of multicategory questions were combined. Factual questions tended to also be questions with two rather than multiple response categories. More study is needed to isolate the effects of these two factors more clearly.

Web based health surveys: Using a Two Step Heckman model to examine their potential for population health analysis

“…Social science needs to find models that are able to robustly examine these new data sources. This is particularly true of web based surveys that appeal to the public interests; one has little control of who decides to respond to an online survey as the majority of web surveys allow participants to self-select into the sample (Schonlau, 2004). As many online surveys do not seek to meet externally defined demographic, socio-economic or spatial quotas based on national registers, the sample representativeness of these surveys needs to be tested before the data may be used for research.…”

“…The majority of web surveys allow participants to self-select into the sample (Schonlau et al, 2004;Bethlehem, 2010). Respondents are those people who happen to visit the website and then decide to participate in the survey (Bethlehem, 2010).…”

How Risky Are Social Networking Sites? A Comparison of Places Online Where Youth Sexual Solicitation and Harassment Occurs

“…Propensity scores statistically adjust for respondents' propensity to be online by comparing data on lifestyle factors with data collected on the same items from samples of random-digit dial telephone surveys. 18,21,29,30 Potential differences in youth characteristics were tested for statistical significance using the Pearson 2 statistic corrected for the survey design with Rao's second-order correction converted into an F statistic. Tables 1-4 are weighted as described above.…”

“…The sample was obtained from the Harris Poll Online opt-in panel, 19 which is consistently comparable to data obtained from random telephone samples of the general populations when appropriate propensity and sample weights are applied. [20][21][22][23] Children were recruited through their parents who were members of the Harris Poll Online. Adults previously indicating having children in the household were randomly invited to participate, stratified by gender and age.…”

Feasibility of using web surveys to collect time–activity data

“…The strength of web surveys is that they do not require an interviewer, which could significantly reduce researchers' labor requirements if a large number of surveys need to be conducted and bring about substantial cost savings compared with telephone surveys (Schonlau et al, 2004;Chang and Krosnick, 2009). Interviewers, even when trained and supervised, are also known to introduce some errors and biases in the data collection process, due to wording and habits as well as some careless behavior (Kiecker and Nelson, 1996;Chang and Krosnick, 2009).…”