Application of Binary Searching for Item Exposure Control in Cognitive Diagnostic Computerized Adaptive Testing

Zheng, Chanjin; Wang, Chun

doi:10.1177/0146621617707509

Cited by 20 publications

(36 citation statements)

References 31 publications

(60 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Incorporating an item exposure method into the item selection algorithm could guarantee limited item exposure rates as well as a more even use of the item pool. C. Wang, Chang, and Huebner (2011) and Zheng and Wang (2017) can be consulted for studies implementing item exposure control in the CD-CAT context. The present article focused on unrestricted algorithms and CAT designs.…”

Section: Discussionmentioning

confidence: 99%

Computerized Adaptive Testing for Cognitively Based Multiple-Choice Data

Yigit

Sorrel

Torre

2018

Applied Psychological Measurement

View full text Add to dashboard Cite

Cognitive diagnosis models (CDMs) are latent class models that hold great promise for providing diagnostic information about student knowledge profiles. The increasing use of computers in classrooms enhances the advantages of CDMs for more efficient diagnostic testing by using adaptive algorithms, referred to as cognitive diagnosis computerized adaptive testing (CD-CAT). When multiple-choice items are involved, CD-CAT can be further improved by using polytomous scoring (i.e., considering the specific options students choose), instead of dichotomous scoring (i.e., marking answers as either right or wrong). In this study, the authors propose and evaluate the performance of the Jensen-Shannon divergence (JSD) index as an item selection method for the multiple-choice deterministic inputs, noisy ''and'' gate (MC-DINA) model. Attribute classification accuracy and item usage are evaluated under different conditions of item quality and test termination rule. The proposed approach is compared with the random selection method and an approximate approach based on dichotomized responses. The results show that under the MC-DINA model, JSD improves the attribute classification accuracy significantly by considering the information from distractors, even with a very short test length. This result has important implications in practical classroom settings as it can allow for dramatically reduced testing times, thus resulting in more targeted learning opportunities.

show abstract

Section: Discussionmentioning

confidence: 99%

Computerized Adaptive Testing for Cognitively Based Multiple-Choice Data

Yigit

Sorrel

Torre

2018

Applied Psychological Measurement

View full text Add to dashboard Cite

show abstract

“…It should be noted that the DBS method, as the NPS method, also benefits from the randomization mechanism (Georgiadou, Triantafillou, & Economides, 2007) for item exposure control. Inspired by an IRT-based CAT technique (Chang & Ying, 1999), Zheng and Wang (2017) modified the method by stratifying the item bank according to item discrimination (Rupp, Templin, & Henson, 2010), and proposed the stratified dynamic binary searching (SDBS) method for fixed-length CD-CAT. G-DINA model discrimination index (GDI).…”

Section: Parametric Item Selection Methods In Cd-catmentioning

confidence: 99%

Nonparametric CAT for CD in Educational Settings With Small Samples

Chang

Chiu

Tsai

2018

Applied Psychological Measurement

View full text Add to dashboard Cite

Cognitive diagnostic computerized adaptive testing (CD-CAT) has been suggested by researchers as a diagnostic tool for assessment and evaluation. Although model-based CD-CAT is relatively well researched in the context of large-scale assessment systems, this type of system has not received the same degree of research and development in small-scale settings, such as at the course-based level, where this system would be the most useful. The main obstacle is that the statistical estimation techniques that are successfully applied within the context of a large-scale assessment require large samples to guarantee reliable calibration of the item parameters and an accurate estimation of the examinees' proficiency class membership. Such samples are simply not obtainable in course-based settings. Therefore, the nonparametric item selection (NPS) method that does not require any parameter calibration, and thus, can be used in small educational programs is proposed in the study. The proposed nonparametric CD-CAT uses the nonparametric classification (NPC) method to estimate an examinee's attribute profile and based on the examinee's item responses, the item that can best discriminate the estimated attribute profile and the other attribute profiles is then selected. The simulation results show that the NPS method outperformed the compared parametric CD-CAT algorithms and the differences were substantial when the calibration samples were small.

show abstract

“…To evaluate the psychometric properties, efficiency, reliability, concurrent validity, and predictive validity of CAT-IA, two CAT simulation studies were carried out. A CAT study is generally composed of six parts: the item bank, item response models, selection methods of initial items, evaluation methods of latent trait, item selection methods, and the stopping rules (Weiss and Kingsbury, 1984). First, the 59-item bank of CAT-IA was established, and the item parameters were estimated under the selected IRT model.…”

Section: Methodsmentioning

confidence: 99%

Development of a Computerized Adaptive Testing for Internet Addiction

et al. 2019

View full text Add to dashboard Cite

Internet addiction disorder has become one of the most popular forms of addiction in psychological and behavioral areas, and measuring it is growing increasingly important in practice. This study aimed to develop a computerized adaptive testing to measure and assess internet addiction (CAT-IA) efficiently. Four standardized scales were used to build the original item bank. A total of 59 polytomously scored items were finally chosen after excluding 42 items for failing the psychometric evaluation. For the final 59-item bank of CAT-IA, two simulation studies were conducted to investigate the psychometric properties, efficiency, reliability, concurrent validity, and predictive validity of CAT-IA under different stopping rules. The results showed that (1) the final 59 items met IRT assumptions, had high discrimination, showed good item-model fit, and were without DIF; and (2) the CAT-IA not only had high measurement accuracy in psychometric properties but also sufficient efficiency, reliability, concurrent validity, and predictive validity. The impact and limitations of CAT-IA were discussed, and several suggestions for future research were provided.

show abstract

Application of Binary Searching for Item Exposure Control in Cognitive Diagnostic Computerized Adaptive Testing

Cited by 20 publications

References 31 publications

Computerized Adaptive Testing for Cognitively Based Multiple-Choice Data

Computerized Adaptive Testing for Cognitively Based Multiple-Choice Data

Nonparametric CAT for CD in Educational Settings With Small Samples

Development of a Computerized Adaptive Testing for Internet Addiction

Contact Info

Product

Resources

About