Dual‐Objective Item Selection Criteria in Cognitive Diagnostic Computerized Adaptive Testing

Abstract

The development of cognitive diagnostic‐computerized adaptive testing (CD‐CAT) has provided a new perspective for gaining information about examinees' mastery on a set of cognitive attributes. This study proposes a new item selection method within the framework of dual‐objective CD‐CAT that simultaneously addresses examinees' attribute mastery status and overall test performance. The new procedure is based on the Jensen‐Shannon (JS) divergence, a symmetrized version of the Kullback‐Leibler divergence. We show that the JS divergence resolves the noncomparability problem of the dual information index and has close relationships with Shannon entropy, mutual information, and Fisher information. The performance of the JS divergence is evaluated in simulation studies in comparison with the methods available in the literature. Results suggest that the JS divergence achieves parallel or more precise recovery of latent trait variables compared to the existing methods and maintains practical advantages in computation and item pool usage.

A Note on the Relationship of the Shannon Entropy Procedure and the Jensen–Shannon Divergence in Cognitive Diagnostic Computerized Adaptive Testing

“…Similar results have been observed by Kang et al (2017) within the framework of dual-objective CD-CAT (Kang et al, 2017;McGlohen & Chang, 2008;Wang et al, 2014;Zheng et al, 2018). For simultaneously estimating examinees' α i and general ability θ i , the dual-objective CD-CAT is based on a CDM and an item response theory model.…”

“…The JSD of dual-objective CD-CAT was defined as mutual information between the two discrete random variables U ij and Z , where U ij has a mixture distribution between P ju ( ) α and P ju ( ) θ , and Z is the binary indicator variable for each distribution. For detailed information about relationship between the JSD, KL information, and Fisher information, please refer to Kang et al (2017).…”

“…Tatsuoka & Ferguson, 2003;Xu et al, 2003Xu et al, , 2016. The third is based on the mutual information, including the expected mutual information index (Wang, 2013) and the Jensen-Shannon divergence (JSD) index (Kang et al, 2017;Minchen & de la Torre, 2016;Yigit et al, 2018). There are other indices, such as the generalized deterministic inputs, noisy "and" gate (G-DINA; de la Torre, 2011) model discrimination index (GDI; Kaplan et al, 2015), the rate function approach (Liu et al, 2015), the halving algorithm (C. Tatsuoka & Ferguson, 2003; W. Y.…”

New Item Selection Method Accommodating Practical Constraints in Cognitive Diagnostic Computerized Adaptive Testing: Maximum Deviation and Maximum Limitation Global Discrimination Indexes

“…CD-CAT, which has the characteristics of a tailor-made test, is promising and will be influential in future educational practices. CD-CAT has received an increasing scholarly attention worldwide (Kang et al, 2017;Huebner et al, 2018).…”

Flexible Computerized Adaptive Tests to Detect Misconceptions and Estimate Ability Simultaneously

“…Existing studies of using CAT to deliver dual-purpose assessments mainly focus on investigating different item selection algorithms (e.g., Cheng & Chang, 2007; Dai et al, 2016; Kang et al, 2017; McGlohen & Chang, 2008; C. Wang et al, 2012, 2014).…”