Linear Models for Optimal Test Design

Linden, Willem J. van der

doi:10.1007/0-387-29054-0

Cited by 236 publications

(254 citation statements)

References 0 publications

Supporting

Mentioning

252

Contrasting

Unclassified

Order By: Relevance

“…Administering test on computers also facilitates delivery of individualized items by providing different difficulty levels, content emphases, and possibility of immediate feedback to the test takers. [37] Excluding the items with overlapping content and answer curing (enemies [38] ) is another area in which these measures could be used to enhance the item delivery and development practices because as the item bank grows the human ability is restricted to compare and detected the enemies. The other promising application is statistical precalibration of the generated items, [39] that uses the concept of item families to develop a statistical model for calibrating siblings (i.e., generated items) based on the commonality within generated items.…”

Section: Discussionmentioning

confidence: 99%

Information-based methods for evaluating the semantics of automatically generated test items

Latifi

Gierl

Ren

et al. 2016

AIR

View full text Add to dashboard Cite

Multiple-choice questions are the popular type of test items that are used for testing the knowledge of health-science students in north America and elsewhere. The motivation of this article is to present the recent advances in the automatic item generation (AIG) and to propose a novel unsupervised approach that extends the information-based Compositional Distributional Semantic Model (CDSM) to measure the semantic relatedness among the pool of automatically generated items. We have used operational item bank from the medical science domain for developing the CDSM and demonstrated our approach using the concepts from AIG research. We illustrated our approach using eleven item models from the medical education domain, and discussed the possible applications to advance the AIG research.

show abstract

Section: Discussionmentioning

confidence: 99%

Information-based methods for evaluating the semantics of automatically generated test items

Latifi

Gierl

Ren

et al. 2016

AIR

View full text Add to dashboard Cite

show abstract

“…Bireyselleştirilmiş testlerde çok boyutlulukla ilgili ilk çalışmalar Bloxom ve Vale (1987), Fan ve Shu (1996), Luecht (1996), Segall (1996) ile başlamış daha sonraları van der Linden (1999;2005), Mudler ve van der Linden (2009) ile devam etmiştir. Yapılan ilk çalışmalar yetenek kestirimleri ve madde seçme yöntemleri üzerine yoğunlaşmıştır.…”

Section: çOk Boyutlu Bireyselleştirilmiş Bilgisayarlı Testlerunclassified

Basit ve Karmaşık Test Desenlerinde Çok Boyutlu Madde Seçme Yöntemlerinin Karşılaştırılması

Özberk

Gelbal

2017

Eğitimde Ve Psikolojide Ölçme Ve Değerlendirme Dergisi

View full text Add to dashboard Cite

ÖzBu araştırmada diğer araştırmaların aksine toplam yetenek puanları gerçek test koşullarına uygun olacak şekilde farklı test koşullarında karşılaştırılmıştır (basit ve karmaşık). Araştırmada test deseni, boyut başına düşen soru sayısı, boyutlar arası korelasyon ve madde seçme yöntemleri olmak üzere dört koşul manipüle edilmiştir. Veri setleri, üretilen madde ve yetenek parametreleri ve M3PL telafi edici çok boyutlu madde tepki kuramı modeli kullanılarak belirlenen korelasyonlara bağlı kalarak üretilmiştir. Çok boyutlu bireyselleştirilmiş bilgisayarlı test uygulamaları sonucu elde edilen toplam yetenek puanları mutlak yanlılık (ABSBIAS), korelasyon ve hata kareleri ortalamasının karekökü (RMSE) kullanılarak karşılaştırılmıştır. Sonuçlar incelendiğinde çok boyutlu test deseni, boyut başına düşen madde sayısı ve boyutlar arası korelasyon değişkenlerinin toplam puanları kestirmede madde seçme yöntemleri üzerinde etkilerinin olduğu belirlenmiştir. Basit yapıdaki bir test için Minimum Hata Varyansı madde seçme yönteminin hem uzun hem de kısa testler için en düşük mutlak yanlılık değerinin ürettiği belirlenmiştir. Model karmaşıklaştıkça Kullback-Leibler madde seçme yönteminin diğer iki yöntemden daha iyi performans gösterdiği belirlenmiştir.Anahtar Kelimeler: Madde seçme yöntemi, çok boyutlu bireyselleştirilmiş bilgisayarlı test, çok boyutlu maddde tepki kuramı, toplam puan kestirimi AbstractIn contrast with the previous studies, this study employed various test designs (simple and complex) which allow the evaluation of the overall ability score estimations across multiple real test conditions. In this study, four factors were manipulated, namely the test design, number of items per dimension, correlation between dimensions and item selection methods. Using the generated item and ability parameters, dichotomous item responses were generated in by using M3PL compensatory multidimensional IRT model with specified correlations. MCAT composite ability score accuracy was evaluated using absolute bias (ABSBIAS), correlation and the root mean square error (RMSE) between true and estimated ability scores. The results suggest that the multidimensional test structure, number of item per dimension and correlation between dimensions had significant effect on item selection methods for the overall score estimations. For simple structure test design it was found that V1 item selection has the lowest absolute bias estimations for both long and short tests while estimating overall scores. As the model gets complex KL item selection method performed better than other two item selection method.Keywords: Item selection method, multidimensional computer adaptive testing, multidimensional item response theory, composite score estimation GİRİŞEğitim ve psikoloji alanında değerlendirme araçlarının başlıca amacı ölçülen özeliğin miktarını belirlemek ve elde edilen numerik puanları kullanarak bireyleri örtük özelliklerine göre sıralamaktır. Puanlar, sıralama amacıyla kullanıldığı durumlarda önemli bir değerlendirme ölçütü olarak olabilmektedi...

show abstract

“…ECD’s approach to specifying valid assemblies is best understood in terms of the problem of assembling an optimal test form from a bank of items using combinatorial optimization (van der Linden, 2005). This approach starts by identifying a target or objective function that essentially states how much evidence is needed (for each variable in the multidimensional case).…”

Section: A Brief Review Of Evidence-centered Assessment Designmentioning

confidence: 99%

How Task Features Impact Evidence From Assessments Embedded in Simulations and Games

Almond¹,

Kim²,

Velasquez³

et al. 2014

Measurement: Interdisciplinary Research and Perspectives

View full text Add to dashboard Cite

One of the key ideas of evidence-centered assessment design (ECD) is that task features can be deliberately manipulated to change the psychometric properties of items. ECD identifies a number of roles that task-feature variables can play, including determining the focus of evidence, guiding form creation, determining item difficulty and discrimination, characterizing proficiency, and producing task variants. Assessment developers can use these task features to manipulate the psychometric properties of both conventional assessment formats and complex tasks embedded in simulations and games.Simulations and games tasks present additional challenges: even defining what corresponds to an item can be difficult. Often task features are determined by game or simulation logic rather than psychometric design. Despite these difficulties, the roles for task features identified in ECD are useful for analyzing the psychometric properties of embedded assessments in simulations and games. This article compares a conventional test of mathematics-problem-solving ability using word problems to an assessment of conceptual physics, creativity, and conscientiousness embedded in the game Newton’s Playground, describing how the task roles play out in each setting and how they can be used to manipulate the evidence provided by an assessment.

show abstract

Linear Models for Optimal Test Design

Cited by 236 publications

References 0 publications

Information-based methods for evaluating the semantics of automatically generated test items

Information-based methods for evaluating the semantics of automatically generated test items

Basit ve Karmaşık Test Desenlerinde Çok Boyutlu Madde Seçme Yöntemlerinin Karşılaştırılması

How Task Features Impact Evidence From Assessments Embedded in Simulations and Games

Contact Info

Product

Resources

About