Diagnostic Classification Models for Ordinal Item Responses

Abstract

The purpose of this study is to develop and evaluate two diagnostic classification models (DCMs) for scoring ordinal item data. We first applied the proposed models to an operational dataset and compared their performance to an epitome of current polytomous DCMs in which the ordered data structure is ignored. Findings suggest that the much more parsimonious models that we proposed performed similarly to the current polytomous DCMs and offered useful item-level information in addition to option-level information. We then performed a small simulation study using the applied study condition and demonstrated that the proposed models can provide unbiased parameter estimates and correctly classify individuals. In practice, the proposed models can accommodate much smaller sample sizes than current polytomous DCMs and thus prove useful in many small-scale testing scenarios.

A general diagnostic classification model for rating scales

“…In both models, all theR values were smaller than 1.02, suggesting convergence to the stationary distribution. For the estimation of the RSDM, we used priors of N(0, 20) for each item parameter and Dirichlet(2) for each attribute profile, similar to Liu and Jiang (2018). These priors are considered less informative and have been recommended in similar DCM studies, such as Chen, Culpepper, Chen, and Douglas (2018) and Jiang and Carter (2018).…”

“…We will briefly compare ten DCMs that are currently capable of handling polytomous item responses, to point out where the RSDM sits in the big picture. These models are (1) the NRDM, (2) the PC-DINA, (3) the general diagnostic model (GDM;von Davier, 2005), (4) the polytomous log-linear cognitive diagnosis model (P-LCDM; Hansen, 2013), (5) the sequential G-DINA model (SG-DINA; Ma & de la Torre, 2016), (6) the DINA model for graded data (DINA-GD; Tu et al, 2017), (7) the general polytomous diagnosis model (GPDM; Chen & de la Torre, 2018), (8) the ordinal response diagnostic model (ORDM; Liu & Jiang, 2018), (9) the modified ordinal response diagnostic model (MORDM; Liu & Jiang, 2018), and (10) the RSDM. We categorize the ten models in Table 14, on the basis of their reduced binary models and polytomous extension features.…”

Using the iterative latent-class analysis approach to improve attribute accuracy in diagnostic classification models

“…Recently the DCM literature has developed beyond binary-scale studies; for example, Chen and de la Torre (2013) described models for polytomous attributes, and Ma and de la Torre 2016proposed a sequential cognitive diagnosis model for polytomous item designs. Therefore, future ILCA works can focus on extending the technique to polytomous designs and investigate how it performs when it comes to an ordinal scale (Liu & Jiang, 2018). In addition, features such as response times and population-level variables in multiple contexts are not considered here (Jiao, Zhan, Liao, & Man, 2018).…”

Using Diagnostic Classification Models in Psychological Rating Scales

“…5If a test has 40 such items, we need to estimate 40 (items) × 5 (response options) × 2 (intercept and main effect) = 400 parameters under the NRDM, which is a lot. To address that problem, (Liu & Jiang, 2018 proposed three smaller DCMs for ordinal item responses: the rating scale diagnostic model (RSDM), the ordinal response diagnostic model (ORDM), and the modified ordinal response diagnostic model (MORDM). These three models are constrained versions of the NRDM with fewer parameters that need to be freely estimated.…”