Abstract:The purpose of this paper is to review some of the key literature on response time as it has played a role in cognitive ability measurement, providing a historical perspective as well as covering current research. We discuss the speed-level distinction, dimensions of speed and level in cognitive abilities frameworks, speed-accuracy tradeoff, approaches to addressing speed-accuracy tradeoff, analysis methods, particularly item response theory-based, response time models from cognitive psychology (ex-Gaussian function, and the diffusion model), and other uses of response time in testing besides ability measurement. We discuss several new methods that can be used to provide greater insight into the speed and level aspects of cognitive ability and speed-accuracy tradeoff decisions. These include item-level time limits, the use of feedback (e.g., CUSUMs), explicit scoring rules that combine speed and accuracy information (e.g., count down timing), and cognitive psychology models. We also review some of the key psychometric advances in modeling speed and level, which combine speed and ability measurement, address speed-accuracy tradeoff, allow for distinctions between response times on items responded to correctly and incorrectly, and integrate psychometrics with information-processing modeling. We suggest that the application of these models and tools is likely to advance both the science and measurement of human abilities for theory and applications.

Existing studies of mediation models have been limited to normal-theory maximum likelihood (ML). Because real data in the social and behavioral sciences are seldom normally distributed and often contain outliers, classical methods generally lead to inefficient or biased parameter estimates. Consequently, the conclusions from a mediation analysis can be misleading. In this article, we propose 2 approaches to alleviate these problems. One is to identify cases that strongly affect testing of mediation using local influence methods or robust methods. The other is to use robust methods for parameter estimation and subsequently test the mediated effect based on the robust estimates. The application of these 2 approaches is illustrated using 1 simulated and 2 real data examples. The interest in 1 real data set is the relationship among marital conflict, children's emotional insecurity, and children's internalizing problems. The other example is concerned with whether ethnic identity mediates the effect of family cohesion on Korean language fluency. Results show that local influence and robust methods rank the influence of cases similarly, but robust methods are more objective. Moreover, when the normality assumption is violated, robust methods give estimates with smaller standard errors and more reliable tests of the mediated effect compared with normal-theory ML. An R program that implements the local influence and robust procedures for mediation analysis is also provided.

Dynamic factor analysis models with time-varying parameters offer a valuable tool for evaluating multivariate time series data with time-varying dynamics and/or measurement properties. We use the Dynamic Model of Activation proposed by Zautra and colleagues (Zautra, Potter, & Reich, 1997) as a motivating example to construct a dynamic factor model with vector autoregressive relations and time-varying cross-regression parameters at the factor level. Using techniques drawn from the state-space literature, the model was fitted to a set of daily affect data (over 71 days) from 10 participants who had been diagnosed with Parkinson's disease. Our empirical results lend partial support and some potential refinement to the Dynamic Model of Activation with regard to how the time dependencies between positive and negative affects change over time. A simulation study is conducted to examine the performance of the proposed techniques when (a) changes in the time-varying parameters are represented using the true model of change, (b) supposedly time-invariant parameters are represented as time-varying, and

There is an increasing market demand to reduce score reporting time. For example, due to a longstanding desire of test takers, the Praxis R teacher certification program reduced the time taken to report scores on multiple choice tests from 22 to 17 business days (ETS News, 2009). Reducing score reporting time is considered as part of a testing program's effort to enhance service to its customers. Preequating, a process to obtain the raw-to-scale score conversion of a form before it is administered intact (Kolen & Brennan, 2004, p. 205), is one way to reduce score reporting time. To make preequating possible, all operational items in the new form must have been previously administered. Then, equating is conducted on the pretested data to obtain conversions of the new form. Two factors are important for a successful preequating. One is an appropriate equating method. The other is that the statistics relevant for equating are invariant across pretesting and operational samples.Item response theory (IRT) has been essential to preequating. Eignor (1985) and Eignor and Stocking (1986) examined the feasibility of using IRT calibrated item pools to preequate new SAT forms. They concluded that preequating was inadequate for the SAT Mathematics test and provided reasons such as multidimensionality and difference in examinees groups. Bejar and Wingersky (1982), who examined the feasibility of preequating the Test of Standard Written English (TSWE), concluded that preequating does not appear to present problems beyond those inherent to IRT equating. Despite these mixed results, IRT preequating continues to be one of the most sought-after methods for preequating.

Equating of tests composed of both discrete and passage‐based multiple choice items using the nonequivalent groups with anchor test design is popular in practice. In this study, we compared the effect of discrete and passage‐based anchor items on observed score equating via simulation. Results suggested that an anchor with a larger proportion of passage‐based items, more items in each passage, and/or a larger degree of local dependence among items within one passage produces larger equating errors, especially when the groups taking the new form and the reference form differ in ability. Our findings challenge the common belief that an anchor should be a miniature version of the tests to be equated. Suggestions to practitioners regarding anchor design are also given.