Measurement Effects of Survey Mode on the Equivalence of Attitudinal Rating Scale Questions

Klausch, Thomas; Hox, Joop J.; Schouten, Barry

doi:10.1177/0049124113500480

Cited by 64 publications

(39 citation statements)

References 64 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In sum, these results reflect theoretical and empirical arguments in the literature that self-administered and interviewer modes often form a dichotomy with respect to measurement bias (De Leeuw 1992, 2008Klausch et al 2013), but also suggest that there may be exceptions to the rule and mode effects remain question dependent phenomena. A question-specific evaluation of effects is, therefore, necessary for any mixed-mode survey.…”

Section: Conclusion For the Cvssupporting

confidence: 83%

Evaluating Bias of Sequential Mixed-mode Designs Against Benchmark Surveys

Klausch

Schouten

Hox

2015

Sociological Methods & Research

Self Cite

View full text Add to dashboard Cite

This study evaluated three types of bias-total, measurement, and selection bias (SB)-in three sequential mixed-mode designs of the Dutch Crime Victimization Survey: telephone, mail, and web, where nonrespondents were followed up face-to-face (F2F). In the absence of true scores, all biases were estimated as mode effects against two different types of benchmarks. In the single-mode benchmark (SMB), effects were evaluated against a F2F reference survey. In an alternative analysis, a ''hybrid-mode benchmark'' (HMB) was used, where effects were evaluated against a mix of the measurements of a web survey and the SB of a F2F survey. A special reinterview design made available additional auxiliary data exploited in estimation for a range of survey variables. Depending on the SMB and HMB perspectives, a telephone, mail, or web design with a F2F follow-up (SMB) or a design involving only mail and/or web but not a F2F followup (HMB) is recommended based on the empirical findings.

show abstract

Section: Conclusion For the Cvssupporting

confidence: 83%

Evaluating Bias of Sequential Mixed-mode Designs Against Benchmark Surveys

Klausch

Schouten

Hox

2015

Sociological Methods & Research

Self Cite

View full text Add to dashboard Cite

show abstract

“…These choices were left up to the country teams and thus reflect country differences in best practices as well as in terms of budgetary opportunities and constraints. There is a growing body of literature (e.g., Klausch, Hox and Schouten 2013;Schouten et al 2013) showing that the choice of survey mode influences the answering patterns of respondents, and if countries use different modes then observed country differences in answers could reflect mode differences rather than genuine country differences.…”

Section: Fieldworkmentioning

confidence: 99%

Generations and Gender Programme Wave 1 data collection

Fokkema

Kveder²,

Hiekel

et al. 2016

DemRes

View full text Add to dashboard Cite

BACKGROUNDThe Generations and Gender Survey (GGS) was developed to stimulate the study of a broad range of topics of relevance to population scientists. So far, at least one wave of the GGS has been conducted in 19 countries. If scholars want to use the GGS for comparative purposes, it is essential that there be cross-national equivalence in terms of survey implementation and representativeness. OBJECTIVESThe two main goals are (1) to describe the main features of the implementation of the GGS in participating countries, and (2) to describe and evaluate the quality of the data collection of the GGS in terms of its cross-sectional representativeness. METHODSWe use weighted and unweighted GGS data for 18 countries and compare this to country-specific information. RESULTSThe quality of sampling and fieldwork procedures of the GGS is generally good. On average, response rates in the GGS are comparable to those in other cross-national surveys. After weighting, the data are generally representative in terms of age, gender, region, and household size, but less so for marital status and educational attainment. Implications for future waves of the GGS are discussed.

show abstract

“…Due to differential nonresponse, different types of respondents tend to end up in the different modes, even in randomized mode experiments. If these differences in sample composition across modes coexist with mode effects, this leads to confounding of substantive and methodological effects (Klausch et al, 2013; Vannieuwenhuyze and Loosveldt, 2013). For example, assume that we use a mixed mode web–interview survey to study drinking behavior.…”

Section: Introductionmentioning

confidence: 99%

Measurement equivalence in mixed mode surveys

2015

Self Cite

View full text Add to dashboard Cite

Surveys increasingly use mixed mode data collection (e.g., combining face-to-face and web) because this controls costs and helps to maintain good response rates. However, a combination of different survey modes in one study, be it cross-sectional or longitudinal, can lead to different kinds of measurement errors. For example, respondents in a face-to-face survey or a web survey may interpret the same question differently, and might give a different answer, just because of the way the question is presented. This effect of survey mode on the question-answer process is called measurement mode effect. This study develops methodological and statistical tools to identify the existence and size of mode effects in a mixed mode survey. In addition, it assesses the size and importance of mode effects in measurement instruments using a specific mixed mode panel survey (Netherlands Kinship Panel Study). Most measurement instruments in the NKPS are multi-item scales, therefore confirmatory factor analysis (CFA) will be used as the main analysis tool, using propensity score methods to correct for selection effects. The results show that the NKPS scales by and large have measurement equivalence, but in most cases only partial measurement equivalence. Controlling for respondent differences on demographic variables, and on scale scores from the previous uni-mode measurement occasion, tends to improve measurement equivalence, but not for all scales. The discussion ends with a review of the implications of our results for analyses employing these scales.

show abstract

Measurement Effects of Survey Mode on the Equivalence of Attitudinal Rating Scale Questions

Cited by 64 publications

References 64 publications

Evaluating Bias of Sequential Mixed-mode Designs Against Benchmark Surveys

Evaluating Bias of Sequential Mixed-mode Designs Against Benchmark Surveys

Generations and Gender Programme Wave 1 data collection

Measurement equivalence in mixed mode surveys

Contact Info

Product

Resources

About