On the complementarity of classical test theory and item response models: item difficulty estimates and computerized adaptive testing

Abstract

This study aims to provide statistical evidence of the complementarity between classical test theory and item response models for certain educational assessment purposes. Such complementarity might support, at a reduced cost, future development of innovative procedures for item calibration in adaptive testing. Classical test theory and the generalized partial credit model are applied to tests comprising multiple choice, short answer, completion, and open response items scored partially. Datasets are derived from the tests administered to the Portuguese population of students enrolled in the 4 th and 6 th grades. The results show a very strong association between the estimates of difficulty obtained from classical test theory and item response models, corroborating the statistical theory of mental testing.

The Effect of Test Length on the Accuracy of Estimating Ability Parameter in the Two- and Three-Parameter Logistic Models: Comparison by Using the Bayesian Method of Expected Prior Mode and Maximum Likelihood Estimation

“…Lord (1953) introduced the principles of this theory, which amends the weaknesses in the classical theory by introducing a method of terminology that features fixed item parameters. These include the difficulty parameter, discrimination, and guessing, in addition to standardizing measurements of difficulty and testees' ability so that test developers can select the most suitable items to classify the testees on the basis of performance levels (Sulaiman, 2009); Costa and Ferrão (Costa & Ferrão, 2015). In the item response theory, the first step is specifying the item parameter and the ability parameter.…”

Web application for the analysis of assessment tests

“…Considering that in the digital world, any conceptual assessment framework faces two main challenges: (a) the complexity of knowledge, capacities and skills to be assessed; (b) the increasing usability of computer and web-based assessments, which requires innovative approaches to the development, delivery and scoring of tests, Ferrão and Prata (2014;2015) explore the adoption of computerized adaptive testing (CAT), aiming at reducing the test size, and simultaneously controlling the impact of such reduction upon the measurement error, in other words, at the production of tests so structured as to generate results which reflect faithfully the degree of knowledge acquisition of the students. In Costa & Ferrão (2015) the authors conceptually present three essential modules of a CAT platform -Informatics (procedures related to test delivery and data collection), Statistical methods (procedures related to data modelling, scoring and calibration) and Topic contents (Items bank and procedures related to items bank manager) -upon which the Adaptive Test Developer operates. There are two statistical approaches to the analysis of the tests, both as a whole, as well as on a question/item basis: the classical test theory (CTT) and the item response theory (IRT) (Hambleton, Swaminathan, & Rogers, 1991).…”