Improved Performance of Model Fit Indices with Small Sample Sizes in Cognitive Diagnostic Models

Tzou, Huey-Ing; Yang, Ya-Huei

doi:10.21449/ijate.482005

Cited by 2 publications

(2 citation statements)

References 21 publications

(26 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Results indicated that performance of fit indices appeared to increase as the sample size increased. Tzou and Yang (2019) also compared the performance of model fit indices in CDMs using small sample sizes (i.e., 50, 75, 100, and 200) and showed that AIC (Akaike, 1974) performed better than other indices. Similarly, Hu et al (2016) evaluated model fit for CDMs using sample sizes of 200, 500, and 1,000 and showed that performances of the three relative fit statistics AIC, BIC (Schwarz, 1978), and CAIC (Bozdogan, 1987) improved when sample size increased.…”

Section: Introductionmentioning

confidence: 99%

Sample Size Requirements for Applying Diagnostic Classification Models

Şen

Cohen

2021

Front. Psychol.

View full text Add to dashboard Cite

Results of a comprehensive simulation study are reported investigating the effects of sample size, test length, number of attributes and base rate of mastery on item parameter recovery and classification accuracy of four DCMs (i.e., C-RUM, DINA, DINO, and LCDMREDUCED). Effects were evaluated using bias and RMSE computed between true (i.e., generating) parameters and estimated parameters. Effects of simulated factors on attribute assignment were also evaluated using the percentage of classification accuracy. More precise estimates of item parameters were obtained with larger sample size and longer test length. Recovery of item parameters decreased as the number of attributes increased from three to five but base rate of mastery had a varying effect on the item recovery. Item parameter and classification accuracy were higher for DINA and DINO models.

show abstract

Section: Introductionmentioning

confidence: 99%

Sample Size Requirements for Applying Diagnostic Classification Models

Şen

Cohen

2021

Front. Psychol.

View full text Add to dashboard Cite

show abstract

“…The study's findings revealed that when item discrimination, sample size, and number of items increased, the estimation accuracy for attributes improved, and when the number of attributes and the degree of the misspecification for a Q-matrix were large, the estimation accuracy deteriorated. Yang et al (2019) investigated how fit indices performed in small sample sizes (i.e. 50, 75, 100, and 200) and confirmed that Akaike information criterion (AIC) is an appropriate choice for model selection under small sample conditions.…”

Section: Introductionmentioning

confidence: 86%

Assessing the Performance of Diagnostic Classification Models in Small Sample Contexts with Different Estimation Methods

Oka

Okada²

2021

Preprint

View full text Add to dashboard Cite

Fueled by the call for formative assessments, diagnostic classification models (DCMs) have recently gained popularity in psychometrics. Despite their potential for providing diagnostic information that aids in classroom instruction and students' learning, empirical applications of DCMs to classroom assessments have been highly limited. This is partly because how DCMs with different estimation methods perform in small sample contexts is not yet wellexplored. Hence, this study aims to investigate the performance of respondent classification and item parameter estimation with a comprehensive simulation design that resembles classroom assessments using different estimation methods. The key findings are the following:(1) although the marked difference in respondent classification accuracy was not observed among the maximum likelihood (ML), Bayesian, and nonparametric methods, the Bayesian method provided slightly more accurate respondent classification in parsimonious DCMs than the ML method, and in complex DCMs, the ML method yielded the slightly better result than the Bayesian method; (2) while item parameter recovery was poor in both Bayesian and ML methods, the Bayesian method exhibited unstable slip values owing to the multimodality of their posteriors under complex DCMs, and the ML method produced irregular estimates that appear to be well-estimated due to a boundary problem under parsimonious DCMs.

show abstract

Improved Performance of Model Fit Indices with Small Sample Sizes in Cognitive Diagnostic Models

Cited by 2 publications

References 21 publications

Sample Size Requirements for Applying Diagnostic Classification Models

Sample Size Requirements for Applying Diagnostic Classification Models

Assessing the Performance of Diagnostic Classification Models in Small Sample Contexts with Different Estimation Methods

Contact Info

Product

Resources

About