Comparing Data from Online and Face-to-face Surveys

Duffy, Bobby; Smith, Kate; Terhanian, George; Bremer, John

doi:10.1177/147078530504700602

Cited by 445 publications

(303 citation statements)

References 3 publications

Supporting

Mentioning

279

Contrasting

Unclassified

Order By: Relevance

“…Comparisons between Internet and traditional surveys where the result is known, such as the poll taken immediately prior to an election have found no significant differences in the accuracy of one or the other method, nor any clear biases (15). The potential for a protechnology bias or a bias toward those who are technologically literate is the biggest concern in the present context (16). An additional concern is that in conducting surveys in multiple languages and cultural contexts there may be subtle differences in language that may, for example, account for why respondents might appear more knowledgeable in some countries on a particular issue.…”

Section: Survey Designmentioning

confidence: 96%

American Exceptionalism? Similarities and Differences in National Attitudes Toward Energy Policy and Global Warming

Reiner

Curry

Figueiredo

et al. 2006

Environ. Sci. Technol.

110

View full text Add to dashboard Cite

Despite sharp differences in government policy, the views of the U.S. public on energy and global warming are remarkably similar to those in Sweden, Britain, and Japan. Americans do exhibit some differences, placing lower priority on the environment and global warming, and with fewer believing that "global warming has been established as a serious problem and immediate action is necessary". There also remains a small hard core of skeptics (<10%) who do not believe in the science of climate change and the need for action, a group that is much smaller in the other countries surveyed. The similarities are, however, pervasive. Similar preferences are manifest across a wide range of technology and fuel choices, in support of renewables, in research priorities, in a basic understanding of which technologies produce or reduce carbon dioxide (or misunderstandings in the case of nuclear power), and in willingness to pay for solving global warming.

show abstract

Section: Survey Designmentioning

confidence: 96%

American Exceptionalism? Similarities and Differences in National Attitudes Toward Energy Policy and Global Warming

Reiner

Curry

Figueiredo

et al. 2006

Environ. Sci. Technol.

110

View full text Add to dashboard Cite

show abstract

“…Schonlau et al (2007) found that web-related attitudinal and behavioral questions complete those about demographics efficiently to produce better representative samples. We found that weighting adjustments fail to erase all differences between online and offline survey results (Duffy et al, 2005;Taylor, 2005;Malhorta and Krosnick, 2007;Pedrazza et al, 2007;Loosveldt and Sonck, 2008;Scherpenzeel and Bethlehem, 2010).…”

Section: Introductionmentioning

confidence: 81%

Using experts’ consensus (the Delphi method) to evaluate weighting techniques in web surveys not based on probability schemes

Toepoel

Emerson

2017

Mathematical Population Studies

View full text Add to dashboard Cite

Weighting techniques in web surveys based on no probability schemes are devised to correct biases due to self-selection, undercoverage, and nonresponse. In an interactive panel, 38 survey experts addressed weighting techniques and auxiliary variables in web surveys. Most of them corrected all biases jointly and applied calibration and propensity score adjustments. Although they claimed that sociodemographic and web-related variables are the most useful auxiliary variables to employ in adjustments, they considered only sociodemographic variables to correct biases because of their availability.

show abstract

“…First, accumulating evidence shows that data collected by means of different modes of data collection may exhibit differential scale usage (Chang & Krosnick, 2009;Dillman, et al, 2009;Duffy, Smith, Terhanian, & Bremer, 2005;Fricker, Galesic, Tourangeau, & Yan, 2005;Roster, Rogers, Albaum, & Klein, 2004;Weijters, et al, 2008). For instance, it is not very surprising that telephone respondents who interact with an interviewer and verbally respond to auditory stimuli use scale points differently than online respondents who get to see a visual response scale in front of them to which they respond by clicking the appropriate response.…”

Section: Discussionmentioning

confidence: 99%

The calibrated sigma method: An efficient remedy for between-group differences in response category use on Likert scales

Weijters

Baumgartner

Geuens

2016

International Journal of Research in Marketing

View full text Add to dashboard Cite

The authors propose a procedure, labeled the calibrated sigma method, which is designed to correct for between-group differences in endorsement likelihood of response categories that are unrelated to the content of the items. The method is especially useful in cross-cultural research where group differences may reflect variation in scale usage rather than substantive differences. However, the procedure is also relevant in other situations, for example, when different data collection modes or different experimental manipulations affect respondents' perception of the meaning of the scale labels. The calibrated sigma method uses information derived from heterogeneous control items (calibration items) to reweight the responses to substantive items in a group-specific way. The advantages of the calibrated sigma method are that it avoids the arbitrariness in the assignment of particular numerical values to response categories; that it is compatible with the linear model, which is used by most marketing researchers; and that it does not require the use of complex nonlinear models involving the estimation of many additional measurement model parameters. The authors validate the calibrated sigma method on a simulated cross-linguistic data set pertaining to 12 different languages; an empirical data set collected from respondents of the same nationality but from two different language groups; and an experimental data set consisting of responses to two different response scale formats. The findings demonstrate that the proposed procedure controls for artefactual scale use differences across groups but does not eliminate substantive differences. It is particularly efficient for marketing research agencies, panel providers and other marketing researchers who analyze surveys involving multiple language groups, different scale formats, multiple modes of data collection, or different manipulations affecting the meaning of the response category labels.

show abstract

Comparing Data from Online and Face-to-face Surveys

Cited by 445 publications

References 3 publications

American Exceptionalism? Similarities and Differences in National Attitudes Toward Energy Policy and Global Warming

American Exceptionalism? Similarities and Differences in National Attitudes Toward Energy Policy and Global Warming

Using experts’ consensus (the Delphi method) to evaluate weighting techniques in web surveys not based on probability schemes

The calibrated sigma method: An efficient remedy for between-group differences in response category use on Likert scales

Contact Info

Product

Resources

About