Imputation Methods to Deal With Missing Responses in Computerized Adaptive Multistage Testing

Cetin-Berber, Dee D.; Sarı, Halil İbrahim; Huggins-Manley, Anne Corinne

doi:10.1177/0013164418805532

Cited by 9 publications

(13 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Extensive reviews of the literature on missing data analysis were performed by Graham (2009), Graham (2012), Schafer (1997), Schafer and Graham (2002), Sinharay, Stern, and Russell (2001), and Vriens and Sinharay (2006). In addition, researchers have considered a wide variety of problems regarding missing data in educational or psychological measurement (e.g., Cetin‐Berber, Sari, & Huggins‐Manley, 2019; De Ayala, Plake, & Impara, 2001; Finch, 2008; Holman & Glas, 2005; Huisman & Molenaar, 2001; Smits, Mellenbergh, & Vorst, 2002; Sijtsma & van der Ark, 2003; Xiao & Bulut, 2020). A brief summary of most of these studies is included in the first subsection of the Supporting Information.…”

Section: Introductionmentioning

confidence: 99%

Score Reporting for Examinees with Incomplete Data on Large‐Scale Educational Assessments

Sinharay

2020

Educational Measurement

View full text Add to dashboard Cite

Technical difficulties occasionally lead to missing item scores and hence to incomplete data on computerized tests. It is not straightforward to report scores to the examinees whose data are incomplete due to technical difficulties. Such reporting essentially involves imputation of missing scores. In this paper, a simulation study based on data from three educational tests is used to compare the performances of six approaches for imputation of missing scores. One of the approaches, based on data mining, is the first application of its kind to the problem of imputation of missing data. The approach based on data mining and a multiple imputation approach based on chained equations led to the most accurate imputation of missing scores, and hence to most accurate score reporting. A simple approach based on linear regression performed the next best overall. Several recommendations are made regarding the reporting of scores to examinees with incomplete data.

show abstract

Section: Introductionmentioning

confidence: 99%

Score Reporting for Examinees with Incomplete Data on Large‐Scale Educational Assessments

Sinharay

2020

Educational Measurement

View full text Add to dashboard Cite

show abstract

“…As the item selection procedure in CAT is dependent on examinees’ observed performance on administered items, the missingness of responses is often seen as missing at random (Han & Guo, 2014; Mislevy, 2016). The mean substitution method is one of the appropriate imputation techniques for missing at random (Huisman, 2000; Cetin‐Berber et al., 2019). Item mean substitution uses the average of an item's observed scores to replace every missing score of this item; person mean substitution uses the average of an examinee's observed scores to replace all of this examinee's missing scores.…”

Section: A New Item Selection Algorithmmentioning

confidence: 99%

Item Selection Algorithm Based on Collaborative Filtering for Item Exposure Control

Pan,

Livne,

Wollack

et al. 2023

Educational Measurement

View full text Add to dashboard Cite

In computerized adaptive testing, overexposure of items in the bank is a serious problem and might result in item compromise. We develop an item selection algorithm that utilizes the entire bank well and reduces the overexposure of items. The algorithm is based on collaborative filtering and selects an item in two stages. In the first stage, a set of candidate items whose expected performance matches the examinee's current performance is selected. In the second stage, an item that is approximately matched to the examinee's observed performance is selected from the candidate set. The expected performance of an examinee on an item is predicted by autoencoders. Experiment results show that the proposed algorithm outperforms existing item selection algorithms in terms of item exposure while incurring only a small loss in measurement precision.

show abstract

“…For MNAR data, the likelihood of a missing response cannot be explained by any measurable variables but is caused by unmeasured variable(s) (e.g., missingness depends on the individual’s ability). Missing data handling approaches have shown to perform differently based on the type of missing mechanisms (e.g., Cetin-Berber et al, 2019; Finch, 2010; Robitzsch & Rupp, 2009). Mislevy and Wu (1996) suggested considering different missing data mechanisms when estimating parameters for obtaining more accurate results.…”

Section: Missing Mechanismsmentioning

confidence: 99%

Evaluating the Effects of Missing Data Handling Methods on Scale Linking Accuracy

Kim

Westine

2022

Educational and Psychological Measurement

View full text Add to dashboard Cite

For large-scale assessments, data are often collected with missing responses. Despite the wide use of item response theory (IRT) in many testing programs, however, the existing literature offers little insight into the effectiveness of various approaches to handling missing responses in the context of scale linking. Scale linking is commonly used in large-scale assessments to maintain scale comparability over multiple forms of a test. Under a common-item nonequivalent group design (CINEG), missing data that occur to common items potentially influence the linking coefficients and, consequently, may affect scale comparability, test validity, and reliability. The objective of this study was to evaluate the effect of six missing data handling approaches, including listwise deletion (LWD), treating missing data as incorrect responses (IN), corrected item mean imputation (CM), imputing with a response function (RF), multiple imputation (MI), and full information likelihood information (FIML), on IRT scale linking accuracy when missing data occur to common items. Under a set of simulation conditions, the relative performance of the six missing data treatment methods under two missing mechanisms was explored. Results showed that RF, MI, and FIML produced less errors for conducting scale linking whereas LWD was associated with the most errors regardless of various testing conditions.

show abstract

Imputation Methods to Deal With Missing Responses in Computerized Adaptive Multistage Testing

Cited by 9 publications

References 33 publications

Score Reporting for Examinees with Incomplete Data on Large‐Scale Educational Assessments

Score Reporting for Examinees with Incomplete Data on Large‐Scale Educational Assessments

Item Selection Algorithm Based on Collaborative Filtering for Item Exposure Control

Evaluating the Effects of Missing Data Handling Methods on Scale Linking Accuracy

Contact Info

Product

Resources

About