The Effect of Item Pools of Different Strengths on the Test Results of Computerized-Adaptive Testing

Abstract

Item response theory provides various important advantages for exams carried out or to be carried out digitally. For computerized adaptive tests to be able to make valid and reliable predictions supported by IRT, good quality item pools should be used. This study examines how adaptive test applications vary in item pools which consist of items with varying difficulty levels. Within the scope of the study, the impact of items was examined where the parameter b differentiates while the parameters a and c are kept in fixed range. To this end, eight different 2000people item pools were designed in simulation which consist of 500 items with ability scores and varying difficulty levels. As a result of CAT simulations, RMSD, BIAS and test lengths were examined. At the end of the study, it was found that tests run by item pools with parameter b in the range that matches the ability level end up with fewer items and have a more accurate stimation. When parameter b takes value in a narrower range, estimation of ability for extreme ability values that are not consistent with parameter b required more items. It was difficult to make accurate estimations for individuals with high ability levels especially in test applications conducted with an item pool that consists of easy items, and for individuals with low ability levels in test applications conducted with an item pool consisting of difficult items.

Pelibatan Alat Peraga Matematika Realistik terhadap Peningkatan Hasil Belajar Siswa di Sekolah Dasar

“…Namun, ada beberapa kekurangan dan kelemahan dengan metode ini. (Kezer, 2021). Namun, ada beberapa kekurangan dan kelemahan dengan metode ini.…”

“…Namun, ada beberapa kekurangan dan kelemahan dengan metode ini. (Kezer, 2021). Paper Based Test (PBT) merupakan tes yang paling sering digunakan untuk penilaian di sekolah sebelumnya (Bagus, 2013).…”

The Effect of Strong and Weak Unidimensional Item Pools on Computerized Adaptive Classification Testing

“…Generally, the CACT studies explain the number of items in the item pool as a pool characteristic that is reasonable but inadequate. Since the distrubition of the Test Information Function (TIF) and/or item parameters are important for CAT and CACT, the information about these features should be explained too (Gündeğer & Doğan, 2018;Kezer, 2021;Thompson, 2009;Thompson, 2011). Additionally, item banks' unidimensionality (as weak or strong unidimensionality) is a specific part of these characteristics, which is seldom studied in the literature compared to other characteristics but affects the results significantly since the item selection and classification criteria have the assume of it.…”