An Evaluation of the Effect of Simplifying Group Structure when Multiple Group IRT Models were Used

Abstract

Consider the situation when a set of new items are administered as the part of several forms including anchor items. In this case, in order to add the new items to the existing item bank, we could either 1) estimate the item parameters of the items in each form and equate them using the information contained in the anchor items (separate calibration), or 2) combine the new item response data to the old data set and calibrate whole the items again using the multiple group IRT (concurrent calibration). However, as the number of groups becomes large, the existing multiple group IRT calibration program fails to converge due to the lack of available memory. In this study we examined the proper method to merge groups in order to avoid the above problem. Simulation studies showed that the separate calibration estimates were closest to the true value, and the concurrent calibration with reduced groups was similar to the separate calibration if the mean values of latent ability were similar among the merged groups.