The Impact of Omitted Responses on the Accuracy of Ability Estimation in Item Response Theory

Ayala, R. J. De; Plake, Barbara S.; Impara, James C.

doi:10.1111/j.1745-3984.2001.tb01124.x

Cited by 52 publications

(96 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Indeed, researchers have addressed specifically how one might handle missing item responses, and how missing data impacts parameter estimation in the item response context, as well as the estimation of statistics such as coefficient alpha (Finch, 2008;Enders, 2004;DeMars, 2002;De Ayala, Plake, & Impara, 2001;Mislevy & Wu, 1996, 1988Lord, 1983Lord, , 1974. These researchers found that estimation of both item response parameters and person ability parameters is compromised when missing data are treated as incorrect (Finch, 2008;DeMars, 2002;De Ayala, Plake, & Impara, 2001;Lord, 1974). Furthermore, even when missing data are treated as not reached, or left as missing, parameter estimation can be detrimentally impacted, particularly the standard errors of the estimates (Finch, 2008;DeMars, 2002).…”

Section: Methods For Dealing With Missing Datamentioning

confidence: 99%

The Use of Multiple Imputation for Missing Data in Uniform DIF Analysis: Power and Type I Error Rates

Finch

2011

Applied Measurement in Education

View full text Add to dashboard Cite

Methods of uniform differential item functioning (DIF) detection have been extensively studied in the complete data case. However, less work has been done examining the performance of these methods when missing item responses are present. Research that has been done in this regard appears to indicate that treating missing item responses as incorrect can lead to inflated Type I error rates (false detection of DIF). The current study builds on this prior research by investigating the utility of multiple imputation methods for missing item responses, in conjunction with standard DIF detection techniques. Results of the study support the use of multiple imputation for dealing with missing item responses. The article concludes with a discussion of these results for multiple imputation in conjunction with other research findings supporting its use in the context of item parameter estimation with missing data.Missing data are a common problem in many data analysis contexts, including those associated with item response data. For example, respondents may simply overlook one or more items on an instrument and neglect to provide a response. In other cases, individuals may not respond to an item for some systematic reason, such as being unsure of the answer to a test question. A number of methods Correspondence should be addressed to Holmes Finch,

show abstract

Section: Methods For Dealing With Missing Datamentioning

confidence: 99%

The Use of Multiple Imputation for Missing Data in Uniform DIF Analysis: Power and Type I Error Rates

Finch

2011

Applied Measurement in Education

View full text Add to dashboard Cite

show abstract

“…In that case, the fact that the item response was missing had a direct relationship to the value that it would have been MNAR. MAR data were also simulated borrowing from approaches used in previous research (Finch, 2008;Enders, 2004;de Ayala, 2001). First, the number of correct item responses was calculated for each simulee across all items in the complete dataset.…”

Section: Methodsmentioning

confidence: 99%

“…Two types of missing data were simulated in the current study, MAR and MNAR. For MNAR data, the requisite proportion of missing values from the items were randomly selected from among those item responses that were simulated to be incorrect, using a method that is common in this literature (Andreis & Ferrari, 2012;Finch, 2008;de Ayala, 2001). This would correspond to the real world situation in which an individual did not respond to an item because the answer was unknown.…”

Section: Methodsmentioning

confidence: 99%

“…Prior research has examined the performance of a number of techniques for handling missing data in the context of IRT. Among the methods that have been studied including listwise deletion (removing from the sample individuals who have missing item responses), scoring the missing responses as incorrect, and scoring the missing responses as fractionally correct, has shown to lead to biased item parameter estimates (e.g., Finch, 2008;De Ayala et al, 2001;Lord, 1974;Ludlow and O'leary, 1999). Other missing data approaches that were also examined in the context of IRT include: corrected item mean substitution imputation, response function imputation, and MIJM.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Recursive Partitioning Methods for Data Imputation in the Context of Item Response Theory: A Monte Carlo Simulation

Edwards

Finch

2018

Psicológica Journal

View full text Add to dashboard Cite

Missing data is a common problem faced by psychometricians and measurement professionals. To address this issue, there are a number of techniques that have been proposed to handle missing data regarding Item Response Theory. These methods include several types of data imputation methods -corrected item mean substitution imputation, response function imputation, multiple imputation, and the EM algorithm, as well as approaches that do not rely on the imputation of missing values -treating the item as not presented, coding missing responses as incorrect, or as fractionally correct. Of these methods, even though multiple imputation has demonstrated the best performance in prior research, higher MAE was still present. Given this higher model parameter estimation MAE for even the best performing missing data methods, this simulation study's goal was to explore the performance of a set of potentially promising data imputation methods based on recursive partitioning. Results of this study demonstrated that approaches that combine multivariate imputation by chained equations and recursive partitioning algorithms yield data with relatively low estimation MAE for both item difficulty and item discrimination. Implications of these findings are discussed.

show abstract

“…When this approach is not feasible, imputation of missing data appears as an advisable alternative. Imputation of missing data in IRT has been studied in the context of unidimensional models (Ayala, Plake, & Impara, 2001;DeMars, 2003;Finch, 2008Finch, , 2011Huisman & Molenaar, 2001;Sijtsma & Van der Ark, 2003). Recently, Wolkowitz and Skorupski (2013) proposed a single imputation approach intended to estimate statistical properties of items but not factor scores.…”

Section: * Introductionmentioning

confidence: 99%

Multiple Imputation of missing values in exploratory factor analysis of multidimensional scales: estimating latent trait scores

Lorenzo‐Seva

Ginkel

2016

AN PSICOL-SPAIN

View full text Add to dashboard Cite

Título: Imputación múltiple de valores perdidos en el análisis factorial exploratorio de escalas multidimensionales: estimación de las puntuaciones de rasgos latentes. Resumen: Los investigadores con frecuencia se enfrentan a la difícil tarea de analizar las escalas en las que algunos de los participantes no han respondido a todos los ítems. En este artículo nos centramos en el análisis factorial exploratorio de escalas multidimensionales (es decir, escalas que constan de varias de subescalas), donde cada subescala se compone de una serie de ítems de tipo Likert, y el objetivo del análisis es estimar las puntuaciones de los participantes en los rasgos latentes correspondientes. En este contexto, se propone un nuevo enfoque para hacer frente a las respuestas faltantes que se basa en (1) la imputación múltiple de las respuestas faltantes y (2) la rotación simultánea de las muestras de datos imputados. Se ha aplicado el método en una muestra de datos reales en que las respuestas que faltantes fueron introducidas artificialmente siguiendo un patrón real de respuestas faltantes, y un estudio de simulación basado en conjuntos de datos artificiales. Los resultados muestran que nuestro enfoque (en concreto, Hot-Deck de imputación múltiple seguido de rotación Consensus Promin) es capaz de calcular correctamente la puntuación factorial estimada incluso para los participantes que tienen valores perdidos. Palabras clave: Valores perdidos; Imputación Hot-Deck; Imputación Predictive mean matching; Imputación múltiple; Consensus Rotation; Puntuaciones factoriales; Análisis factorial exploratorio. Abstract:Researchers frequently have to analyze scales in which some participants have failed to respond to some items. In this paper we focus on the exploratory factor analysis of multidimensional scales (i.e., scales that consist of a number of subscales) where each subscale is made up of a number of Likert-type items, and the aim of the analysis is to estimate participants' scores on the corresponding latent traits. We propose a new approach to deal with missing responses in such a situation that is based on (1) multiple imputation of non-responses and (2) simultaneous rotation of the imputed datasets. We applied the approach in a real dataset where missing responses were artificially introduced following a real pattern of non-responses, and a simulation study based on artificial datasets. The results show that our approach (specifically, Hot-Deck multiple imputation followed of Consensus Promin rotation) was able to successfully compute factor score estimates even for participants that have missing data.

show abstract

The Impact of Omitted Responses on the Accuracy of Ability Estimation in Item Response Theory

Cited by 52 publications

References 12 publications

The Use of Multiple Imputation for Missing Data in Uniform DIF Analysis: Power and Type I Error Rates

The Use of Multiple Imputation for Missing Data in Uniform DIF Analysis: Power and Type I Error Rates

Recursive Partitioning Methods for Data Imputation in the Context of Item Response Theory: A Monte Carlo Simulation

Multiple Imputation of missing values in exploratory factor analysis of multidimensional scales: estimating latent trait scores

Contact Info

Product

Resources

About