Nonparametric Classification Method for Multiple-Choice Items in Cognitive Diagnosis

Wang, Yu; Chiu, Chia-Yi; Köhn, Hans Friedrich

doi:10.3102/10769986221133088

Cited by 4 publications

(1 citation statement)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Model-based estimation techniques, such as Expectation Maximization (EM) and Markov Chain Monte Carlo (MCMC), are commonly employed to fit DCM to educational test data [9]. However, these methods require large sample sizes to achieve stable and reliable parameter estimates [10]. They are also vulnerable to model misspecification, and are sensitive to the quality of test items or the correctness of the Q-matrix specification [11,12].…”

Section: Introductionmentioning

confidence: 99%

Supervised diagnostic classification of cognitive attributes using data augmentation

Yoon,

Gweon,

Yoo

2024

PLoS ONE

View full text Add to dashboard Cite

Over recent decades, machine learning, an integral subfield of artificial intelligence, has revolutionized diverse sectors, enabling data-driven decisions with minimal human intervention. In particular, the field of educational assessment emerges as a promising area for machine learning applications, where students can be classified and diagnosed using their performance data. The objectives of Diagnostic Classification Models (DCMs), which provide a suite of methods for diagnosing students’ cognitive states in relation to the mastery of necessary cognitive attributes for solving problems in a test, can be effectively addressed through machine learning techniques. However, the challenge lies in the latent nature of cognitive status, which makes it difficult to obtain labels for the training dataset. Consequently, the application of machine learning methods to DCMs often assumes smaller training sets with labels derived either from theoretical considerations or human experts. In this study, the authors propose a supervised diagnostic classification model with data augmentation (SDCM-DA). This method is designed to utilize the augmented data using a data generation model constructed by leveraging the probability of correct responses for each attribute mastery pattern derived from the expert-labeled dataset. To explore the benefits of data augmentation, a simulation study is carried out, contrasting it with classification methods that rely solely on the expert-labeled dataset for training. The findings reveal that utilizing data augmentation with the estimated probabilities of correct responses substantially enhances classification accuracy. This holds true even when the augmentation originates from a small labeled sample with occasional labeling errors, and when the tests contain lower-quality items that may inaccurately measure students’ true cognitive status. Moreover, the study demonstrates that leveraging augmented data for learning can enable the successful classification of students, thereby eliminating the necessity for specifying an underlying response model.

show abstract