Estimating Diffusion-Based Item Response Theory Models: Exploring the Robustness of Three Old and Two New Estimators

Abstract

Diffusion-based item response theory models for responses and response times in tests have attracted increased attention recently in psychometrics. Analyzing response time data, however, is delicate as response times are often contaminated by unusual observations. This can have serious effects on the validity of statistical inference. In this article, we compare three established and two new estimation approaches for diffusion-based item response theory models with respect to their robustness. The three established approaches are the marginal maximum likelihood (ML) estimator for continuous time, the marginal ML estimator for discrete time, and the weighted least squares (WLS) estimator. The new approaches are two modifications of the WLS estimator with better robustness properties. The performance of the estimators is compared in a simulation study. The simulation study illustrates that the new approaches are robust against some forms of random independent contamination. The marginal ML estimator for discrete time also performs well. The marginal ML estimator for continuous time is heavily affected by contamination.

An Overview of Models for Response Times and Processes in Cognitive Tests

“…Tuerlinckx and De Boeck (2005) have shown that both these cognitive models can be approximately re-parameterized as item response models and thus as measurement models for test data. Since then, van der Maas et al (2011) have developed a version of the diffusion model for cognitive test data (see Ranger and Kuhn, 2018, for estimation methods), and Rouder et al (2015) and Ranger et al (2014), have developed race models for joint response accuracy and response time data from cognitive tests. The diffusion model and the race model as process models are discussed after the hierarchical model is presented.…”

A Multidimensional Partially Compensatory Response Time Model on Basis of the Log-Normal Distribution

“…Before analyzing the data, we screened the data for careless responders. Test takers that respond carelessly in most items provide a series of short response times that distorts the parameter estimates (e.g., Ranger & Kuhn, 2018). We defined responses as careless in case they were given with response times that were 2.5 times the interquartile range below the box of a boxplot.…”

Online Proctoring Based on Item Responses and Response Times

“…where µ I and Σ I each denote the mean vector and variance-covariance matrix of the transformed item parameters. We note that although the study assumed particular models, the procedures suggested below can be applied to other parametric settings (Klein Entink, Fox, & van der Linden, 2009;Molenaar, Tuerlinckx, & van der Maas, 2015;Ranger & Kuhn, 2018). As long as the model allows inference on the latent trait levels, the procedures can be adapted to the presumed model with the use of proper estimators.…”