Fluctuations of Ability Estimates in Testing in Item Response Theory

Abstract

By analyzing the fluctuations of ability estimates in testing, we first obtain the purely probabilistic fluctuations of ability estimates in a one-time testing under the condition that the students' abilities can be estimated by using the item response theory, and next, by taking into account such the probabilistic fluctuations, we find students who reveal the discrepancies of observed abilities between two separated testings. When such discrepancies of abilities are observed, test results are considered to be affected by some factors such as the physical conditions of the examinees, the teacher's teaching skills, and students' study skill developments. To describe such a phenomenon, we proposed a basic formula. The accuracies are obtained under the situation that the observed data follows the item response theory. To investigate whether we can assume such a condition or not, we have introduced the matrix decomposition perspective, and confirmed that the item response theory were used properly. Using an example case took in a university mathematics testing, we have shown how we have extracted the purely probabilistic fluctuations and segregated fluctuations due to other factors.