Multiscale Measurement of Extreme Response Style

Abstract

This article extends a methodological approach considered by Bolt and Johnson for the measurement and control of extreme response style (ERS) to the analysis of rating data from multiple scales. Specifically, it is shown how the simultaneous analysis of item responses across scales allows for more accurate identification of ERS, and more effective control of ERS effects on the substantive trait estimates, than when analyzing just one scale. Moreover, unlike a competing approach presented by Greenleaf, the current strategy can accommodate conditions in which the substantive traits across scales correlate, as is almost always the case in social sciences research. Simulation and real data analyses are used for illustration.Keywords item response theory, extreme response style, nominal response model A well-known limitation of self-report survey instruments in the social sciences is their susceptibility to response style effects. Response styles (or alternatively ''response sets'') refer to stylistic tendencies in how respondents assign ratings to items, such as tendencies to use only a small number of the available rating scale options (Cronbach, 1946). Response styles have the potential to undermine the validity of scale scores and can also interfere with the application of psychometric models (such as item response models) that imply invariance in the functioning of items across respondents.Although various response styles have been documented (see Baumgartner & Steenkamp, 2001, for a review), extreme response style (ERS) is a form that has been the focus of much methodological research. ERS is characterized by a tendency to select the end points of a rating scale, such as 1 ¼ strongly disagree or 7 ¼ strongly

A Simulation Study on Methods of Correcting for the Effects of Extreme Response Style

“…Therefore, v 2i0 = v 2i(m) = 1 and v 2ij = 0 if 0 \ j \ (m). Bolt and Newton (2011) extended this approach by adding a second trait dimension to the model and by modeling the ERS dimension on the combined items from both trait dimensions. According to Bolt and Newton, this method allows a better differentiation between the traits and ERS, which should lead to more accurate trait estimates.…”

“…However, the simulation study by Bolt and Newton (2011) did not investigate the possible influence of test length and sample size on the accuracy of trait recovery. In the present study, these factors will be taken into account.…”

“…We hypothesize that both methods based on trait estimates from item response models will achieve a better trait recovery than regression residuals. Since there are no previous studies evaluating the trait recovery in mixed PCMs and the simulation study on multidimensional models (Bolt & Newton, 2011) only considered a limited number of conditions, the comparison of these two methods is exploratory.…”

“…Last, the implications of these results for the correction of trait estimates for response style effects will be discussed. This study will focus on one response style, namely ERS, because two of the three methods we evaluate have been suggested specifically for this response style (Bolt & Johnson, 2009;Bolt & Newton, 2011;Rost, Carstensen, & von Davier, 1997) and it also appears to be the most important response style in terms of its variance explanation incremental to the trait (Wetzel & Carstensen, 2015).…”

Examining the attitude-achievement paradox in PISA using a multilevel multidimensional IRT model for extreme response style

“…Relative to earlier studies on response styles which generally only verified the existence of response styles, the MIRT model is also capable of correcting ERS bias in providing substantive trait estimates. Bolt and Newton (2011) illustrated the method in relation to PISA 2006 data. Based on the example items in Table 2, Table 3 presents some hypothetical examples of response patterns and the corresponding trait estimates as observed by Bolt & Newton (2011).…”