Evaluating the Effects of Missing Data Handling Methods on Scale Linking Accuracy

Abstract

For large-scale assessments, data are often collected with missing responses. Despite the wide use of item response theory (IRT) in many testing programs, however, the existing literature offers little insight into the effectiveness of various approaches to handling missing responses in the context of scale linking. Scale linking is commonly used in large-scale assessments to maintain scale comparability over multiple forms of a test. Under a common-item nonequivalent group design (CINEG), missing data that occur to common items potentially influence the linking coefficients and, consequently, may affect scale comparability, test validity, and reliability. The objective of this study was to evaluate the effect of six missing data handling approaches, including listwise deletion (LWD), treating missing data as incorrect responses (IN), corrected item mean imputation (CM), imputing with a response function (RF), multiple imputation (MI), and full information likelihood information (FIML), on IRT scale linking accuracy when missing data occur to common items. Under a set of simulation conditions, the relative performance of the six missing data treatment methods under two missing mechanisms was explored. Results showed that RF, MI, and FIML produced less errors for conducting scale linking whereas LWD was associated with the most errors regardless of various testing conditions.

Improving Peak Shift Estimation to Rank Exams by Difficulty

“…Exam results datasets used in IRT often have missing data as a result of candidates avoiding some exams because of difficulty, time limits, or adaptive testing. Various researchers have therefore improved IRT to enable it to be applied to exam data [15]- [21]. However, if we consider selectively omitted exam data as exam results data with missing data, the same problem occurs.…”