Mixture Random-Effect IRT Models for Controlling Extreme Response Style on Rating Scales

Abstract

Respondents are often requested to provide a response to Likert-type or rating-scale items during the assessment of attitude, interest, and personality to measure a variety of latent traits. Extreme response style (ERS), which is defined as a consistent and systematic tendency of a person to locate on a limited number of available rating-scale options, may distort the test validity. Several latent trait models have been proposed to address ERS, but all these models have limitations. Mixture random-effect item response theory (IRT) models for ERS are developed in this study to simultaneously identify the mixtures of latent classes from different ERS levels and detect the possible differential functioning items that result from different latent mixtures. The model parameters can be recovered fairly well in a series of simulations that use Bayesian estimation with the WinBUGS program. In addition, the model parameters in the developed models can be used to identify items that are likely to elicit ERS. The results show that a long test and large sample can improve the parameter estimation process; the precision of the parameter estimates increases with the number of response options, and the model parameter estimation outperforms the person parameter estimation. Ignoring the mixtures and ERS results in substantial rank-order changes in the target latent trait and a reduced classification accuracy of the response styles. An empirical survey of emotional intelligence in college students is presented to demonstrate the applications and implications of the new models.

Validity of Three IRT Models for Measuring and Controlling Extreme and Midpoint Response Styles

“…It is worth noting that the low to medium correlations between the ERS and MRS factors and their criteria do not imply that the models of interest are not valid in measuring ERS and MRS. The short scales in the current research might be insufficient for estimating ERS and MRS because the small number of items might cause high uncertainty (e.g., standard error and classification error) of ERS and MRS estimates in these models (Huang, 2016;Adams et al, 2019). Indeed, the classification performance of the mPCM was low in Study 1, and this might cause the poor efficacy of the mPCM in measuring MRS.…”

Use of Mixed Item Response Theory in Rating Scales

“…According to the distribution of the students in the response categories, LC1 could be named as the acquiescence response category and LC2 as the general class or midpoint response style. Previous researchers (Eid & Zickar, 2007;Huang, 2016) recommended the use of MixIRT to determine response styles. Furthermore, since different response styles may have different response patterns, MixIRT models can be used to classify individuals according to their response styles (Huang, 2016).…”

“…From the studies conducted with MixIRT models in the literature, it can be seen that MixIRT is used in achievement tests (Bolt, Cohen & Wollack, 2001;De Ayala & Santiago, 2017;Yalçın, 2018), Likert-type scales (Egberink et al, 2010;Ölmez & Cohen, 2018), personality questionnaires (Maij-de Meij et al, 2008), and to determine the response style (Eid & Zickar, 2007;Huang, 2016). The literature reveals that more studies were encountered regarding application of MixIRT in the achievement tests but there is a lack of studies on the use and introduction of MixIRT models in graded scales.…”

Effects of calpain inhibitor on the apoptosis of hepatic stellate cells induced by calcium ionophore A23187

“…Calcium ionophore A23187 has been reported to cause cell apoptosis through the nicotinamide adenine dinucleotide phosphate hydrogenase oxidase pathway or mitochondrial permeability transition and can also change intracellular calcium distribution, thus disrupting calcium ion homeostasis . This compound can also cause an ERS response in some cells, but this phenomenon has not yet been examined in HSCs. Calpain inhibitors are tetrapeptide acetal protease inhibitors that behave similarly to intracellular endogenous calpain inhibitory proteins by recognizing and specifically binding to conformational features of calpain associated with Ca 2+ binding, thus reducing calpain activity .…”