Towards a Tailored Hybrid Recommendation-based System for Computerized Adaptive Testing through Clustering and IRT

Silva, Wesley; Spalenza, Marcos; Bourguet, Jean-Rémi; Oliveira, Elias

doi:10.5220/0009419902600268

Cited by 2 publications

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent developments show a merging trend of psychometric and machine learnings (ML), for example, Bergner and colleagues (2012) conducted collaborative filtering analysis through an ML-based IRT; Pliakos and colleagues (2019) adopted ML approaches to assist the interpretability of IRT parameters; Jiang and colleagues (2019b) adopted AdaBoost algorithm from ML family to supplement cognitive diagnosis modeling; Silva and colleagues (2020) integrated clustering approaches to IRT-based computerized adaptive testing. It can be found that the trend takes place mostly in modeling strategies; however, model fit evaluation engrafting from one field to the other is paid less attention.…”

Section: Introductionmentioning

confidence: 99%

Is the Area Under Curve Appropriate for Evaluating the Fit of Psychometric Models?

Han

Zhang

Jiang

et al. 2022

Educational and Psychological Measurement

View full text Add to dashboard Cite

In the literature of modern psychometric modeling, mostly related to item response theory (IRT), the fit of model is evaluated through known indices, such as χ2, M2, and root mean square error of approximation (RMSEA) for absolute assessments as well as Akaike information criterion (AIC), consistent AIC (CAIC), and Bayesian information criterion (BIC) for relative comparisons. Recent developments show a merging trend of psychometric and machine learnings, yet there remains a gap in the model fit evaluation, specifically the use of the area under curve (AUC). This study focuses on the behaviors of AUC in fitting IRT models. Rounds of simulations were conducted to investigate AUC’s appropriateness (e.g., power and Type I error rate) under various conditions. The results show that AUC possessed certain advantages under certain conditions such as high-dimensional structure with two-parameter logistic (2PL) and some three-parameter logistic (3PL) models, while disadvantages were also obvious when the true model is unidimensional. It cautions researchers about the dangers of using AUC solely in evaluating psychometric models.

show abstract