The Effect of Strong and Weak Unidimensional Item Pools on Computerized Adaptive Classification Testing

Abstract

Computerized Adaptive Classification Tests (CACT) aim to classify individuals effectively with high classification accuracy and few items over large item pools. The characteristic features of the item pool include the number of items, item factor loadings, the distribution of the Test Information Function, and dimensionality. In this study, we present the results of a comprehensive simulation study that was examined how item selection methods (MFI-KLI), ability estimation methods (EAP-WLE) and classification methods (SPRT-CI) were affected by strong and weak unidimensional item pools. Findings of the study indicate that CI had always produced results with classification accuracy similar to SPRT but with a test length of almost half. Additionally, KLI and MFI item selection methods were not affected by the item pool characteristic as weak or strong unidimensionality. From findings of this study, it can be recommended to use CI with EAP in CACT studies, whether the item pool is weak or strong unidimensional, but WLE only under strong unidimensional item pools. Additionally, EAP and SPRT methods are recommended to prefer in the weak unidimensional item pool.