The Restricted DINA Model: A Comprehensive Cognitive Diagnostic Model for Classroom-Level Assessments

Abstract

The nonparametric classification (NPC) method has been proven to be a suitable procedure for cognitive diagnostic assessments at a classroom level. However, its nonparametric nature impedes the obtention of a model likelihood, hindering the exploration of crucial psychometric aspects, such as model fit or reliability. Reporting the reliability and validity of scores is imperative in any applied context. The present study proposes the restricted deterministic input, noisy “and” gate (R-DINA) model, a parametric cognitive diagnosis model based on the NPC method that provides the same attribute profile classifications as the nonparametric method while allowing to derive a model likelihood and, subsequently, to compute fit and reliability indices. The suitability of the new proposal is examined by means of an exhaustive simulation study and a real data illustration. The results show that the R-DINA model properly recovers the posterior probabilities of attribute mastery, thus becoming a suitable alternative for comprehensive small-scale diagnostic assessments.

Exploring Approaches for Estimating Parameters in Cognitive Diagnosis Models with Small Sample Sizes

“…The item quality was set to a medium level by setting g j = s j = 0.20. Thus, the generating model coincides with the R-DINA since DINA reduces to R-DINA when ϕ = s j = g j ∀ j [36], with ϕ = 0.20 in this case. It is common to generate item parameters in this way in simulation studies as it will allow the recovery of item parameters to be studied in a simple way [2,42].…”

“…Exploring this prior to interpreting any model will therefore prove crucial. Although Nájera et al (in press) [36] have found that R-DINA can perform better than DINA even when this constraint is violated, in low sample conditions where the estimation of the DINA model is very noisy (i.e., very small sample size, poor item quality), we want to emphasize that it is necessary that, before interpreting the output of R-DINA, an evaluation of its fit to the data and the relative fit with respect to the DINA model is carried out. As noted in the results, when appropriate to use, R-DINA provides accurate classifications, such as nonparametric methods, and an unbiased estimate of reliability.…”

“…The Restricted DINA model (R-DINA) [36] was estimated with the cdmTools package [23]. In the R-DINA model, a single parameter ϕ is estimated for the whole model, which is defined as the proportion of observed responses that depart from the ideal responses.…”

“…Another route is to renounce estimating item parameters in low-sample situations, adopting a nonparametric procedure such as that proposed by Chiu and Douglas (2013) [35], which compares observed responses to ideal response patterns. A recent proposal is to reduce the model as much as possible, estimating a single parameter that accounts for the differences between observed and ideal response patterns [36].…”

Design and implementation of an AI‐enabled visual report tool as formative assessment to promote learning achievement and self‐regulated learning: An experimental study

“…Cognitive diagnosis assesses and tracks students' mastery of specific knowledge and levels of learning in related competencies, which provides learners with accurate and personalized learning support. IRT (Tay et al, 2015) and DINA models (Nájera et al, 2023) are widely used cognitive diagnostic models based on psychometrics. They employ empirical response functions that depict the correlation between a learner's performance in an exercise and their level of understanding.…”