Evaluating Assessment via Item Response Theory Utilizing Information Function With R

Abstract

Item and test information functions that measure the reliability of an assessment via Item response theory (IRT) are described in the paper for the practitioners. While the four parameters binary IRT models are frequently used, their relationship to the precision level is not commonly discussed. More notably, the benefits, limitations, and constraints of the information approach have not been fully examined systematically. On this basis, with useful and practical examples, the paper formally introduces the graphical approach of presenting item and test information functions that could be easily carried out using the irt R package. The simple R syntax is illuminated throughout the text to show the separate item parameter effects and the combinational and offsetting effects on the information when all the item parameters are used in an assessment. In particular, the characteristics of the 4PL information function that have not been paid much attention to are highlighted and illustrated about its functionality and application. The paper ends with a guide on the information approach to the selection of the items and setting up an assessment. The scope, limitations, and constraints of the graphical approach are also discussed.