Fitting IRT Models to Dichotomous and Polytomous Data: Assessing the Relative Model–Data Fit of Ideal Point and Dominance Models

Abstract

This study investigated the relative model-data fit of an ideal point item response theory (IRT) model (the generalized graded unfolding model [GGUM]) and dominance IRT models (e.g., the two-parameter logistic model [2PLM] and Samejima's graded response model [GRM]) to simulated dichotomous and polytomous data generated from each of these models. The relative magnitudes of the adjusted c 2 /df ratios for item pairs and item triples at the test level were used to evaluate fit. Two simulation studies were conducted, one for dichotomous data and the other for polytomous data. Relative fit of the ideal point and dominance models were compared with respect to different conditions: test length, sample size, and sample type. In many simulated conditions, it was found that comparing relative fits (using test-level doubles and triples adjusted c 2 /df ratios) almost always consistently pointed to the correct IRT model. However, GGUM could fit dichotomous two-parameter logistic (2PL) data well when the scale length was short (i.e., 15 items); nevertheless, an examination of estimated GGUM item parameters clearly shows dominance item characteristics. Results of the simulation studies and implications are discussed.

Uncovering curvilinear relationships between conscientiousness and job performance: How theoretically appropriate measurement makes an empirical difference.

“…This is consistent with past research that has shown slightly better fit for the 2-parameter logistic model (a special case of the GRM) compared to the GGUM for doubles and triples for some personality measures when item singles suggest better fit for the GGUM (see Stark et al, 2006). Moreover, as noted previously, the 2 /df ratios Ͻ 3 criterion has been shown to be inappropriate for doubles and triples (Tay et al, 2011). As can be seen in Figure 5, the information function for the GRM and GGUM showed higher reliability at the low end of the continuum versus the high end but relatively low reliability at higher levels.…”

“…Doubles and triples showed means larger than the suggested cutoff of 3, suggesting some potential problems with local dependence, an issue addressed in more detail later in this section. However, it has been recently suggested (Tay, Ali, Drasgow, & Williams, 2011) that the 2 /df ratios Ͻ 3 criterion is inappropriate for doubles and triples, and these results are consistent with past IRT model-data fit analyses of personality measures (e.g., see Stark et al, 2006). Given these results, we concluded that person parameters would be well-interpreted and continued with our substantive analyses.…”

Using item response theory to investigate the structure of anticipated affect: do self-reports about future affective reactions conform to typical or maximal models?

“…As previously stated, the application of dominance IRT models requires reverse scoring, where this is not the case for unfolding models. Following established analytic procedures ( Tay et al, 2011 ) negatively valenced items were reverse coded first.…”

“…We compared the relative fit of the ideal point and dominance models with the adjusted χ 2 /df ( Tay et al, 2011 ).…”

The lz(p)* Person-Fit Statistic in an Unfolding Model Context

“…The generation of model-fitting data closely followed previous research based on the GGUM (Roberts, Donoghue, & Laughlin, 2002;Tay, Ali, Drasgow, & Williams, 2011;Wang, Tay, & Drasgow, 2013). Item scores were randomly drawn from multinomial distributions based on the conditional probabilities given by Equation 1.…”