A Method for Imputing Response Options for Missing Data on Multiple-Choice Assessments

Abstract

When missing values are present in item response data, there are a number of ways one might impute a correct or incorrect response to a multiple-choice item. There are significantly fewer methods for imputing the actual response option an examinee may have provided if he or she had not omitted the item either purposely or accidentally. This article applies the multiple-choice model, a multiparameter logistic model that allows for in-depth distractor analyses, to impute response options for missing data in multiple-choice items. Following a general introduction of the issues involved with missing data, the article describes the details of the multiple-choice model and demonstrates its use for multiple imputation of missing item responses. A simple simulation example is provided to demonstrate the accuracy of the imputation method by comparing true item difficulties ( p values) and item–total correlations ( r values) to those estimated after imputation. Missing data are simulated according to three different types of missing mechanisms: missing completely at random, missing at random, and missing not at random.

Recursive Partitioning Methods for Data Imputation in the Context of Item Response Theory: A Monte Carlo Simulation

“…Other missing data approaches that were also examined in the context of IRT include: corrected item mean substitution imputation, response function imputation, and MIJM. Of these, MIJM has shown perhaps the most promise for use with IRT data, based on simulation research in multiple contexts within IRT (Wolkowitz & Skorupski, 2013;Finch, 2008;). These studies revealed that imputed data based on MIJM exhibited the least amount of item difficulty and discrimination parameter estimation bias, particularly when the data were MAR (Finch, 2008).…”

Multiple Imputation of missing values in exploratory factor analysis of multidimensional scales: estimating latent trait scores

“…This assumption is more acceptable in the context of psychological tests. Wolkowitz and Skorupski (2013) proposed a method for imputing response options for missing data based on multiple-choice assessments but state that it is intended for test development planning purposes, and that additional research is needed before it can be used to operationally score a test.…”

“…Imputation of missing data in IRT has been studied in the context of unidimensional models (Ayala, Plake, & Impara, 2001;DeMars, 2003;Finch, 2008Finch, , 2011Huisman & Molenaar, 2001;Sijtsma & Van der Ark, 2003). Recently, Wolkowitz and Skorupski (2013) proposed a single imputation approach intended to estimate statistical properties of items but not factor scores. Finally, no research has yet been undertaken in the framework of multidimensional IRT.…”

Imputation Methods to Deal With Missing Responses in Computerized Adaptive Multistage Testing

“…However, these studies are in reference to item data that are missing due to the adaptive test not presenting an item to an examinee. In situations of adaptive or nonadaptive tests where examinees are permitted to skip items, item omissions may depend on the examinees’ underlying ability levels, which can be considered MNAR (Wang, Jiao, & Xiang, 2013; Wolkowitz & Skorupski, 2013). Given that the missing data mechanism of skipped items can theoretically take on any form in a specific testing context, we investigate in this study missing data methods in the presence of MCAR, MAR, and MNAR missingness in the ca-MST context.…”

“…For MNAR missingness, the probability of missingness was dependent on a person’s theta value (Wang et al, 2013; Wolkowitz & Skorupski, 2013), such that…”