A comparison of computerized adaptive testing strategies

Kezer, Fatih; Koç, Nizamettin

doi:10.12973/jesr.2014.41.8

Cited by 6 publications

(5 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For computerized adaptive tests to be able to make valid and reliable predictions supported by IRT, good quality item pools should be used (Hambleton, Swaminathan & Rogers, 1991;Weiss, 1985). In adaptive test designs, from 50% to 80% could be saved in test length (Bulut & Kan, 2012;Comert, 2008;Iseri, 2002;Kalender, 2011;Kaptan, 1993;Kezer, 2013;McDonald, 2002;McBride & Martin, 1983;Olsen, Maynes, Slavvson & Ho, 1989;Oztuna, 2008;Scullard, 2007;Smits, Cuijper & Straten, 2011). With CAT, each individual can get a test appropriate for his or her ability level.…”

Section: Discussionmentioning

confidence: 99%

The Effect of Item Pools of Different Strengths on the Test Results of Computerized-Adaptive Testing

Kezer

2021

International Journal of Assessment Tools in Education

Self Cite

View full text Add to dashboard Cite

Item response theory provides various important advantages for exams carried out or to be carried out digitally. For computerized adaptive tests to be able to make valid and reliable predictions supported by IRT, good quality item pools should be used. This study examines how adaptive test applications vary in item pools which consist of items with varying difficulty levels. Within the scope of the study, the impact of items was examined where the parameter b differentiates while the parameters a and c are kept in fixed range. To this end, eight different 2000people item pools were designed in simulation which consist of 500 items with ability scores and varying difficulty levels. As a result of CAT simulations, RMSD, BIAS and test lengths were examined. At the end of the study, it was found that tests run by item pools with parameter b in the range that matches the ability level end up with fewer items and have a more accurate stimation. When parameter b takes value in a narrower range, estimation of ability for extreme ability values that are not consistent with parameter b required more items. It was difficult to make accurate estimations for individuals with high ability levels especially in test applications conducted with an item pool that consists of easy items, and for individuals with low ability levels in test applications conducted with an item pool consisting of difficult items.

show abstract

Section: Discussionmentioning

confidence: 99%

The Effect of Item Pools of Different Strengths on the Test Results of Computerized-Adaptive Testing

Kezer

2021

International Journal of Assessment Tools in Education

Self Cite

View full text Add to dashboard Cite

show abstract

“…When the EAP method was used for proficiency estimation, reliable results were obtained. Some studies that have employed CATs have also concluded that the EAP method gives better results than other techniques (Eroğlu, 2013;Kezer, 2013).…”

Section: Discussionmentioning

confidence: 99%

“…From the findings, it was clear that the examinations can be applied as CATs on the condition that the item bank was enriched with a larger number of specific questions. Subsequent studies on CAT examined the applicability of different proficiency tests that were not administered by the federal government but were taken by large groups (academic achievement, foreign language proficiency, computer literacy) according to different CAT strategies (Gökçe, 2012;Kezer & Koç, 2014;Özbaşı & Demirtaşlı, 2015). Some studies were conducted in this vein in medicine (Öztuna, 2012), and that long surveys given to patients for accurate diagnoses can be employed as CATs.…”

Section: Development Of Computerized Adaptive Version Of Ecodlmentioning

confidence: 99%

Development of a Computerized Adaptive Version of the Turkish Driving Licence Exam

Çikrikçi

Yalçın

Kalender

et al. 2020

International Journal of Assessment Tools in Education

View full text Add to dashboard Cite

This study tested the applicability of the theoretical Examination for Candidates of Driving License (ECODL) in Turkey as a computerized adaptive test (CAT). Firstly, various simulation conditions were tested for the live CAT through an item response theory-based calibrated item bank. The application of the simulated CAT was based on data from e-exams administered by the Ministry of National Education (MoNE). Results of the first stage of the study were used to determine the rules for starting, continuing, and terminating the live CAT exam for ECODL. Secondly, the live CAT exam was applied according to the results of the simulation. Candidate drivers (n = 280) who had taken the ECODL as an e-test participated in the second stage. Thirdly, the opinions of the individuals who took the computer-based test towards the computer-based testing application were mapped. In the termination rule of the CAT-based ECODL, testing with a fixed number of questions yielded the smallest estimated measurement error. We also found that when ECODL was implemented as CAT, it could reliably differentiate among testers in terms of competence of theoretical knowledge of driving and provide basis for accurate decisions regarding their proficiency. According to the findings obtained on the candidates' opinions on the computer-based testing application, it was seen that they considered computer-based application more practical an easier in terms of testing.

show abstract

“…Wang and Wang (2001) state that the ability estimation methods are an important CACT component that affects both the selection of the items proper for the estimated ability level and the termination of the test. In the literature, it is seen that Maximum Likelihood Estimation, Weighted Likelihood Estimation (WLE), Expected a Posteriori (EAP), Maximum a Posteriori methods are frequently examined among the ability estimation methods (Breslow & Holubkov;1997;Cheng & Liou, 2000;Diao & Reckase, 2009;Eggen & Straetmans, 2000;Gökçe, 2012;Kalender, 2011;Kezer, 2013;Penfield & Bergeron, 2005;Tao, Shi & Chang, 2012;Wang, 1997;Wang et al, 1999;Wang & Vispoel, 1998;Wang & Wang, 2001;Warm, 1989;Wouda & Eggen, 2009;Yi, Wang & Ban, 2000). Note that all of these estimation methods make biased estimations to some extent (Warm, 1989).…”

Section: Introductionmentioning

confidence: 99%

The Effect of Strong and Weak Unidimensional Item Pools on Computerized Adaptive Classification Testing

Gündeğer

Soysal

2022

Journal of Teacher Education and Lifelong Learning

View full text Add to dashboard Cite

Computerized Adaptive Classification Tests (CACT) aim to classify individuals effectively with high classification accuracy and few items over large item pools. The characteristic features of the item pool include the number of items, item factor loadings, the distribution of the Test Information Function, and dimensionality. In this study, we present the results of a comprehensive simulation study that was examined how item selection methods (MFI-KLI), ability estimation methods (EAP-WLE) and classification methods (SPRT-CI) were affected by strong and weak unidimensional item pools. Findings of the study indicate that CI had always produced results with classification accuracy similar to SPRT but with a test length of almost half. Additionally, KLI and MFI item selection methods were not affected by the item pool characteristic as weak or strong unidimensionality. From findings of this study, it can be recommended to use CI with EAP in CACT studies, whether the item pool is weak or strong unidimensional, but WLE only under strong unidimensional item pools. Additionally, EAP and SPRT methods are recommended to prefer in the weak unidimensional item pool.

show abstract

A comparison of computerized adaptive testing strategies

Abstract: ÖZET

Cited by 6 publications

References 17 publications

The Effect of Item Pools of Different Strengths on the Test Results of Computerized-Adaptive Testing

The Effect of Item Pools of Different Strengths on the Test Results of Computerized-Adaptive Testing

Development of a Computerized Adaptive Version of the Turkish Driving Licence Exam

The Effect of Strong and Weak Unidimensional Item Pools on Computerized Adaptive Classification Testing

Contact Info

Product

Resources

About