Multidimensional adaptive testing with constraints on test content

Veldkamp, Bernard P.; Linden, Willem J. van der

doi:10.1007/bf02295132

Cited by 95 publications

(130 citation statements)

References 15 publications

Supporting

Mentioning

124

Contrasting

Unclassified

Order By: Relevance

“…Currently, interactive items can be included by pointing to raw HTML stems, although these do not directly integrate with the responses to each item. Additionally, more holistic control over content constraints may be included by providing support for so-called shadow testing designs (e.g., Veldkamp and Linden 2002).…”

Section: Discussionmentioning

confidence: 99%

Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications

Chalmers¹

2016

J. Stat. Soft.

View full text Add to dashboard Cite

Computerized adaptive testing (CAT) is a powerful technique to help improve measurement precision and reduce the total number of items required in educational, psychological, and medical tests. In CATs, tailored test forms are progressively constructed by capitalizing on information available from responses to previous items. CAT applications primarily have relied on unidimensional item response theory (IRT) to help select which items should be administered during the session. However, multidimensional CATs may be constructed to improve measurement precision and further reduce the number of items required to measure multiple traits simultaneously.A small selection of CAT simulation packages exist for the R environment; namely, catR (Magis and Raîche 2012), catIrt (Nydick 2014), and MAT (Choi and King 2014). However, the ability to generate graphical user interfaces for administering CATs in realtime has not been implemented in R to date, support for multidimensional CATs have been limited to the multidimensional three-parameter logistic model, and CAT designs were required to contain IRT models from the same modeling family. This article describes a new R package for implementing unidimensional and multidimensional CATs using a wide variety of IRT models, which can be unique for each respective test item, and demonstrates how graphical user interfaces and Monte Carlo simulation designs can be constructed with the mirtCAT package.

show abstract

Section: Discussionmentioning

confidence: 99%

Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications

Chalmers¹

2016

J. Stat. Soft.

View full text Add to dashboard Cite

show abstract

“…Besides, many models for scoring polytomous items (OSTINI; NERING, 2006) have been presented in the literature. When several abilities account for the response behavior, multidimensional IRT models can be applied (SEGALL, 1996; VELDKAMP; VAN DER LINDEN, 2002;RECKASE, 2009). …”

Section: Computerized Adaptive Testingmentioning

confidence: 99%

“…Item selection Many item selection rules have been proposed for CAT. Maximum Fisher information (BIRNBAUM, 1968) is most commonly applied, but Fisher interval information (VEERKAMP; BERGER,1997), Kullback-Leibler information (CHANG; YING, 1996; VELDKAMP; VAN DER LINDEN, 2002), or mutual information (WEISSMAN, 2007) might be applied as well. All these item selection rules have in common that they try to maximize information obtained about the candidate in order to minimize the error of estimation.…”

Section: Five Basic Steps Of Catmentioning

confidence: 99%

Bayesian computerized adaptive testing

Veldkamp

Matteucci

2013

Ensaio: aval.pol.públ.Educ.

Self Cite

View full text Add to dashboard Cite

Computerized adaptive testing (CAT) comes with many advantages. Unfortunately, it still is quite expensive to develop and maintain an operational CAT. In this paper, various steps involved in developing an operational CAT are described and literature on these topics is reviewed. Bayesian CAT is introduced as an alternative, and the use of empirical priors is proposed for estimating item and person parameters to reduce the costs of CAT. Methods to elicit empirical priors are presented and a two small examples are presented that illustrate the advantages of Bayesian CAT. Implications of the use of empirical priors are discussed, limitations are mentioned and some suggestions for further research are formulated.

show abstract

“…This information is used as a prior distribution to select next item which is believed to contribute to the precision of ability estimates. KL information measures the distance between two likelihoods at true ability and current ability and it is concluded that KL information is a better indicator discriminating true and estimated ability based on posterior densities and doesn't require ability levels close to each other (Veldkamp and van der Linden, 2002). Also KL information overcomes the attenuation paradox which helps to estimate correct θ values rather than using Fisher information.…”

Section: Introductionmentioning

confidence: 99%

Basit ve Karmaşık Test Desenlerinde Çok Boyutlu Madde Seçme Yöntemlerinin Karşılaştırılması

Özberk

Gelbal

2017

Eğitimde Ve Psikolojide Ölçme Ve Değerlendirme Dergisi

View full text Add to dashboard Cite

ÖzBu araştırmada diğer araştırmaların aksine toplam yetenek puanları gerçek test koşullarına uygun olacak şekilde farklı test koşullarında karşılaştırılmıştır (basit ve karmaşık). Araştırmada test deseni, boyut başına düşen soru sayısı, boyutlar arası korelasyon ve madde seçme yöntemleri olmak üzere dört koşul manipüle edilmiştir. Veri setleri, üretilen madde ve yetenek parametreleri ve M3PL telafi edici çok boyutlu madde tepki kuramı modeli kullanılarak belirlenen korelasyonlara bağlı kalarak üretilmiştir. Çok boyutlu bireyselleştirilmiş bilgisayarlı test uygulamaları sonucu elde edilen toplam yetenek puanları mutlak yanlılık (ABSBIAS), korelasyon ve hata kareleri ortalamasının karekökü (RMSE) kullanılarak karşılaştırılmıştır. Sonuçlar incelendiğinde çok boyutlu test deseni, boyut başına düşen madde sayısı ve boyutlar arası korelasyon değişkenlerinin toplam puanları kestirmede madde seçme yöntemleri üzerinde etkilerinin olduğu belirlenmiştir. Basit yapıdaki bir test için Minimum Hata Varyansı madde seçme yönteminin hem uzun hem de kısa testler için en düşük mutlak yanlılık değerinin ürettiği belirlenmiştir. Model karmaşıklaştıkça Kullback-Leibler madde seçme yönteminin diğer iki yöntemden daha iyi performans gösterdiği belirlenmiştir.Anahtar Kelimeler: Madde seçme yöntemi, çok boyutlu bireyselleştirilmiş bilgisayarlı test, çok boyutlu maddde tepki kuramı, toplam puan kestirimi AbstractIn contrast with the previous studies, this study employed various test designs (simple and complex) which allow the evaluation of the overall ability score estimations across multiple real test conditions. In this study, four factors were manipulated, namely the test design, number of items per dimension, correlation between dimensions and item selection methods. Using the generated item and ability parameters, dichotomous item responses were generated in by using M3PL compensatory multidimensional IRT model with specified correlations. MCAT composite ability score accuracy was evaluated using absolute bias (ABSBIAS), correlation and the root mean square error (RMSE) between true and estimated ability scores. The results suggest that the multidimensional test structure, number of item per dimension and correlation between dimensions had significant effect on item selection methods for the overall score estimations. For simple structure test design it was found that V1 item selection has the lowest absolute bias estimations for both long and short tests while estimating overall scores. As the model gets complex KL item selection method performed better than other two item selection method.Keywords: Item selection method, multidimensional computer adaptive testing, multidimensional item response theory, composite score estimation GİRİŞEğitim ve psikoloji alanında değerlendirme araçlarının başlıca amacı ölçülen özeliğin miktarını belirlemek ve elde edilen numerik puanları kullanarak bireyleri örtük özelliklerine göre sıralamaktır. Puanlar, sıralama amacıyla kullanıldığı durumlarda önemli bir değerlendirme ölçütü olarak olabilmektedi...

show abstract

Multidimensional adaptive testing with constraints on test content

Abstract: mathematical programming, multidimensional adaptive testing, multidimensional item response theory, posterior expected Kullback-Leibler information,

Cited by 95 publications

References 15 publications

Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications

Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications

Bayesian computerized adaptive testing

Basit ve Karmaşık Test Desenlerinde Çok Boyutlu Madde Seçme Yöntemlerinin Karşılaştırılması

Contact Info

Product

Resources

About