Credible Comparisons Using Interpersonally Incomparable Data: Nonparametric Scales with Anchoring Vignettes

Wand, Jonathan

doi:10.1111/j.1540-5907.2012.00597.x

Cited by 24 publications

(22 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Different tests, perhaps based on experiments, need to be developed for constructs for which it is difficult to obtain objective measures. If future research confirms the negative results found here, then researchers willing to use vignettes should consider exploiting nonparametric approaches that rest on weaker assumptions (e.g., Wand, 2007). …”

Section: Resultsmentioning

confidence: 64%

Slipping Anchor?: Testing the Vignettes Approach to Identification and Correction of Reporting Heterogeneity

d’Uva¹,

Lindeboom²,

O’Donnell³

et al. 2011

Journal of Human Resources

View full text Add to dashboard Cite

Anchoring vignettes are increasingly used to identify and correct heterogeneity in the reporting of health, work disability, life satisfaction, political efficacy, etc. with the aim of improving interpersonal comparability of subjective indicators of these constructs. The method relies on two assumptions: vignette equivalence -the vignette description is perceived by all to correspond to the same state; and, response consistency -individuals use the same response scales to rate the vignettes and their own situation. We propose tests of these assumptions. For vignette equivalence, we test a necessary condition of no systematic variation with observed characteristics in the perceived difference in states corresponding to any two vignettes. To test response consistency we rely on the assumption that objective indicators fully capture the covariation between the construct of interest and observed individual characteristics, and so offer an alternative way to identify response scales, which can then be compared with those identified from the vignettes. We also introduce a weaker test that is valid under a less stringent assumption. We apply these tests to cognitive functioning and mobility related health problems using data from the English Longitudinal Survey of Ageing. Response consistency is rejected for both health domains according to the first test, but the weaker test does not reject for cognitive functioning. The necessary condition for vignette equivalence is rejected for both health domains. These results cast some doubt on the validity of the vignettes approach, at least as applied to these health domains. JEL Classification: C35, C42, I12

show abstract

Section: Resultsmentioning

confidence: 64%

Slipping Anchor?: Testing the Vignettes Approach to Identification and Correction of Reporting Heterogeneity

d’Uva¹,

Lindeboom²,

O’Donnell³

et al. 2011

Journal of Human Resources

View full text Add to dashboard Cite

show abstract

“…Nearly forty years after its development, political scientists continue to employ A-M scaling to study a variety of political contexts (e.g., Saiegh, 2009;Hollibaugh, Rothenberg and Rulison, 2013;Lo, Proksch and Gschwend, 2014). Gary King and Jonathan Wand have also introduced a DIF-correction method based on anchoring vignettes that is equivalent to a nonparametric version of the A-M model (King and Wand, 2007;Wand, 2013). However, A-M scaling has been an underutilized tool in general the estimation of spatial models of public opinion and in particular to answer the important questions posed above.…”

Section: Introductionmentioning

confidence: 99%

Using Bayesian Aldrich‐McKelvey Scaling to Study Citizens' Ideological Preferences and Perceptions

Hare¹,

Armstrong

Bakker³

et al. 2014

American J Political Sci

136

View full text Add to dashboard Cite

Aldrich-McKelvey scaling is a powerful method that corrects for differentialitem functioning (DIF) in estimating the positions of political stimuli (e.g., parties and candidates) and survey respondents along a latent policy dimension from issue scale data. DIF arises when respondents interpret issue scales (like the standard liberal-conservative scale) differently and distort their placements of the stimuli and themselves. We develop a Bayesian implementation of the classical maximum likelihood Aldrich-McKelvey scaling method that overcomes some important shortcomings in the classical procedure. We then apply this method to study citizens' ideological preferences and perceptions using data from the 2004-2012 American National Election Studies and the 2010 Cooperative Congressional Election Study. Our findings indicate that DIF biases self-placements on the liberal-conservative scale in a way that understates the extent of polarization in the contemporary American electorate and that citizens have remarkably accurate perceptions of the ideological positions of Senators and Senate candidates.

show abstract

“…Figure 2 shows the median SROCC across the data for each domain. 18 For most domains the vignettes appear to work well, with the median correlation assuming values between 0.85 and 0.95. Only the domains ''Confidentiality'' and ''Choice'' appear to have a slightly worse performance, presenting a median correlation that varies between 0.75 and 0.80.…”

Section: Consistent and Near-consistent Ordering Of Vignettesmentioning

confidence: 97%

“…17 The coefficient of variation of the number of alternative orderings is 14.35, while for the number of SROCCS that occur with a frequency greater than 1% it is 0.91. 18 For each domain, we have computed the median SROCC on the basis of tables analogous to Table 4. 19 We are not aware of any study that explicitly defines a threshold of acceptability for the rank order correlation coefficient above which we can assume that vignette equivalence holds.…”

Section: Consistent and Near-consistent Ordering Of Vignettesmentioning

confidence: 99%

“…[3][4][5][6][7]), health system responsiveness [2,[8][9][10], happiness and job satisfaction [11,12], national identity [13] and state effectiveness [14]. Despite the growing popularity of the vignette methodology to address the issue of reporting heterogeneity, the formal evaluation of the validity of the approach remains a topic of research [15][16][17][18]. Two critical assumptions need to hold in order for the method to be valid.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Analysis of the validity of the vignette approach to correct for heterogeneity in reporting health system responsiveness

2010

View full text Add to dashboard Cite

Despite the growing popularity of the vignette methodology to deal with self-reported, categorical data, the formal evaluation of the validity of this methodology is still a topic of research. Some critical assumptions need to hold in order for this method to be valid. In this paper we analyse the assumption of "vignette equivalence" using data on health system responsiveness contained within the World Health Survey. We perform several tests to check the assumption of vignette equivalence. First, we use a test based on the global ordering of the vignettes. A minimal condition for the assumption of vignette equivalence to hold is that individual responses are consistent with the global ordering of vignettes. Secondly, using the hierarchical ordered probit model (HOPIT) model on the pool of countries, we undertake sensitivity analyses, stratifying countries according to the Inglehart-Welzel scale and the Human Development Index. The results of this analysis are robust, suggesting that the vignette equivalence assumption is not contradicted. Thirdly, we model the reporting behaviour of the respondents through a two-step regression procedure to evaluate whether the vignettes construct is perceived by respondents in different ways. Overall, across the analyses the results do not contradict the assumption of vignette equivalence and accordingly lend support to the use of the vignette methodology when analysing self-reported data and health system responsiveness.

show abstract

Credible Comparisons Using Interpersonally Incomparable Data: Nonparametric Scales with Anchoring Vignettes

Cited by 24 publications

References 17 publications

Slipping Anchor?: Testing the Vignettes Approach to Identification and Correction of Reporting Heterogeneity

Slipping Anchor?: Testing the Vignettes Approach to Identification and Correction of Reporting Heterogeneity

Using Bayesian Aldrich‐McKelvey Scaling to Study Citizens' Ideological Preferences and Perceptions

Analysis of the validity of the vignette approach to correct for heterogeneity in reporting health system responsiveness

Contact Info

Product

Resources

About