Multidimensional Modeling of Traits and Response Styles

Abstract

Abstract. Response styles can influence item responses in addition to a respondent’s latent trait level. A common concern is that comparisons between individuals based on sum scores may be rendered invalid by response style effects. This paper investigates a multidimensional approach to modeling traits and response styles simultaneously. Models incorporating different response styles as well as personality traits (Big Five facets) were compared regarding model fit. Relationships between traits and response styles were investigated and different approaches to modeling extreme response style (ERS) were compared regarding their effects on trait estimates. All multidimensional models showed a better fit than the unidimensional models, indicating that response styles influenced item responses with ERS showing the largest incremental variance explanation. ERS and midpoint response style were mainly trait-independent whereas acquiescence and disacquiescence were strongly related to several personality traits. Expected a posteriori estimates of participants’ trait levels did not differ substantially between two-dimensional and unidimensional models when a set of heterogeneous items was used to model ERS. A minor adjustment of trait estimates occurred when the same items were used to model ERS and the trait, though the ERS dimension in this approach only reflected scale-specific ERS, rather than a general ERS tendency.

IRTree models with ordinal and multidimensional decision nodes for response styles and trait‐based rating responses

“…The multidimensional parametrization of node models allows one to specify multiple judgement processes that simultaneously affect decisions among competing response categories, such as joint effects of the target trait and response styles. Ordinal response processes and multidimensional parametrizations of threshold probabilities have previously been accommodated in traditional IRT approaches (e.g., Bolt & Newton, ; Bolt et al ., ; Falk & Cai, ; Johnson & Bolt, ; Wetzel & Carstensen, ) or random‐threshold models (Jin & Wang, ; Wang & Wu, ; Wang et al ., ). Here, we have shown that ordinal response processes and multidimensional node models can also be integrated in the family of IRTree models.…”

“…Different families of stochastic models have been proposed to accommodate response styles in psychological assessments with common Likert response formats. On the one hand, traditional IRT models for ordinal responses such as the partial credit or rating scale model were generalized to finite mixture distribution models (e.g., Austin, Deary, & Egan, ; Eid & Rauber, ; Meiser & Machunsky, ; Rost, ; Wetzel, Carstensen, & Böhnke, ), multidimensional IRT models (e.g., Bolt, Lu, & Kim, ; Bolt & Newton, ; Falk & Cai, ; Johnson & Bolt, ; Morren, Gelissen, & Vermunt, ; Wetzel & Carstensen, ) or random‐threshold models (e.g., Jin & Wang, ; Wang, Wilson, & Shih, ; Wang & Wu, ). These extended IRT models maintain the assumption of an ordinal trait‐based response process, according to which the observed rating responses reflect gradual degrees of agreement with the item content, but account for response styles in terms of additional person parameters or in terms of discrete or continuous distributions of random threshold parameters.…”

A New Model for Acquiescence at the Interface of Psychometrics and Cognitive Psychology

“…It is based on pairs of logical opposite items (e.g., "I am happy" and "I am sad") and ARS is simply the mean across such items (e.g., Soto et al, 2008;Winkler, Kanouse, & Ware, 1982). A third approach focuses only on the agree categories and was used with content-heterogeneous items (e.g., Weijters et al, 2010b) or with content-homogeneous items (e.g., Falk & Cai, 2016;Johnson & Bolt, 2010;Wetzel & Carstensen, 2017). Even though it is obvious that the approaches differ in their theoretical definition and operationalization of ARS, they are seldomly compared, and empirical evidence of convergent validity is mixed (Billiet & McClendon, 2000;Ferrando et al, 2004;Kam & Zhou, 2015).…”

Using a Mixed IRT Model to Assess the Scale Usage in the Measurement of Job Satisfaction

“…For response formats with 2–10 options, the increase of the number of response categories of 2–6 categories only leads to an increase in reliability and convergent validity measured by a heterotrait-monomethod correlation (Lozano et al, 2008; Maydeu-Olivares et al, 2009; Culpepper, 2013). However, ISU is responsible for up to 25% of score variability (Wetzel and Carstensen, 2015) and can thereby make a contribution to the artificial increase of reliability (Weather et al, 2005; Jin and Wang, 2014). A separate assessment of true trait variance and response style variance is necessary to obtain an unbiased reliability measure.…”