Hierarchical Diagnostic Classification Modeling of Reading Comprehension

Abstract

The Hierarchical Diagnostic Classification Model (HDCM) reflects on the sequences of the presentation of the essential materials and attributes to answer the items of a test correctly. In this study, a foreign language reading comprehension test was analyzed employing HDCM and the generalized deterministic-input, noisy and gate (G-DINA) model to determine and compare respondents’ mastery profiles in the test’s predefined skills and to illustrate the relationships among the attributes involved in the test to capture the influence of sequential teaching of materials on increasing the probability of getting an item a correct answer. Furthermore, Differential Item Functioning (DIF) analysis was applied to detect whether the test functions as a reason for the gender gap in participants’ achievement. Finally, classification consistency and accuracy indices are studied. The results showed that the G-DINA and one of the HDCMs fit the data well. However, although the results of HDCM showed the existence of attribute dependencies in the reading comprehension test, the relative fit indices highlight a significant difference between the G-DINA and HDCM, favoring G-DINA. Moreover, results indicate that there is a significant difference between males and females in six items in favor of females. Besides, classification consistency and accuracy indices specify that the Iranian University Entrance Examination holds a 71% chance of categorizing a randomly selected test taker consistently on two distinct test settings and a 78% likelihood of accurately classifying any randomly selected student into the true latent classes. As a result, it can be concluded that the Iranian University Entrance Examination can be considered as a valid and reliable test.

Approximate Invariance Testing in Diagnostic Classification Models in the Presence of Attribute Hierarchies: A Bayesian Network Approach

“…Within the traditional LCDM framework, attributes are assumed to be unstructured in the sense that directional or hierarchical relationships among attributes are not assumed nor specified [16]. However, as [15] notes, hierarchical structures naturally arise in psychological and educational contexts (e.g., [32][33][34]).…”

Approximate Invariance Testing in Diagnostic Classification Models in the Presence of Attribute Hierarchies: A Bayesian Network Approach

“…Within the traditional LCDM framework, attributes are assumed to be unstructured in the sense that directional or hierarchical relationships among attributes are not assumed nor specified . However as Rupp, Templin, and Henson (2010) note, hierarchical structures naturally arise in psychological and educational contexts (e.g., Ma, Wang, & Xiao, 2023;Tabatabaee-Yazdi, 2020;Zhang & Wang, 2020).…”

On the Identifiability of Cognitive Diagnostic Models: Diagnosing Students’ Translation Ability

“…According to the correlation between attributes, CDMs are classified into two types: (1) conjunctive or non-compensatory models, and (2) disjunctive or compensatory models. If the performance of one or more of the attributes can compensate for the non-performance of other attributes, compensatory models (Disjunctive) are used Tabatabaee-Yazdi, 2020). Conversely, if the performance of one or more of the attributes cannot compensate for the non-performance of other attributes, non-compensatory models (Conjunctive), are used.…”

“…Conversely, if the performance of one or more of the attributes cannot compensate for the non-performance of other attributes, non-compensatory models (Conjunctive), are used. Thus, in order to achieve a high probability of a correct answer, performance of all the required attributes is needed Tabatabaee-Yazdi, 2020).…”

“…However, general CDMs do not hypothesize any pre-specified relations between underlying sub-skills. Therefore, several kinds of interactions are possible within the same assessment, assuming different relationships between attributes across the items Tabatabaee-Yazdi, 2020).…”