Evaluating cognitive theory: A joint modeling approach using responses and response times.

Entink, Rinke H. Klein; Kuhn, Jörg‐Tobias; Hornke, Lutz F.; Fox, Jean-Paul

doi:10.1037/a0014877

Cited by 78 publications

(78 citation statements)

References 89 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…First, it is particularly appealing in the context of Borsboom's (2006) view that the fact that measurement models lack substantive foundation is the main reason psychometrics has had a limited impact. That is, although the analysis of choice response time data in a hierarchical framework has already been addressed in the psychometric literature (Klein Entink, Kuhn, Hornke, & Fox, 2009;van der Linden, 2007), our use of a popular process model such as the diffusion model is novel. At the same time, of course, the process interpretation limits the applicability of the present approach, because one is now required to assume that the process assumptions are not violated.…”

Section: Discussionmentioning

confidence: 99%

Hierarchical diffusion models for two-choice response times.

Vandekerckhove

Tuerlinckx

Lee

2011

Psychological Methods

276

324

View full text Add to dashboard Cite

Two-choice response times are a common type of data, and much research has been devoted to the development of process models for such data. However, the practical application of these models is notoriously complicated, and flexible methods are largely nonexistent. We combine a popular model for choice response times-the Wiener diffusion process-with techniques from psychometrics in order to construct a hierarchical diffusion model. Chief among these techniques is the application of random effects, with which we allow for unexplained variability among participants, items, or other experimental units. These techniques lead to a modeling framework that is highly flexible and easy to work with. Among the many novel models this statistical framework provides are a multilevel diffusion model, regression diffusion models, and a large family of explanatory diffusion models. We provide examples and the necessary computer code.Keywords: response time, psychometrics, hierarchical, random effects, diffusion model Supplemental materials: http://dx.doi.org/10.1037/a0021765.suppIn his 1957 presidential address at the 65th annual business meeting of the American Psychological Association, Lee Cronbach drew a captivating sketch of the state of psychology at the time. He focused on the two distinct disciplines that then existed in the field of scientific psychology. On the one side, there was the experimental discipline, which concerned itself with the systematic manipulation of conditions in order to observe the consequences. On the other side, there was the correlational discipline, which focused on the study of preexisting differences between individuals or groups. Cronbach saw many potential contributions of these disciplines to one another and argued that the time and opportunity had come for the two dissociated fields to crossbreed: "We are free at last to look up from our own bedazzling treasure, to cast properly covetous glances upon the scientific wealth of our neighbor discipline. Trading has already been resumed, with benefit to both parties" (Cronbach, 1957, p. 675). Two decades onward, Cronbach (1975) saw the hybrid discipline flourishing across several domains.In the area of measurement of psychological processes, there exists a schism similar to the one Cronbach pointed out in his presidential address. Psychological measurement and individual differences are studied in the domain of psychometrics, whereas cognitive processes are the stuff of the more nomothetic mathematical psychology. In both areas, statistical models are used extensively. There are common models based on the (general) linear model, such as analysis of variance (ANOVA) and regression, but we focus on more advanced, nonlinear techniques.Experimental psychology has, for a long time, made use of process models to describe interesting psychological phenomena in various fields. Some famous examples are Sternberg's (1966) sequential exhaustive search model for visual search and memory scanning, Atkinson and Shiffrin's (1968) multistore model for memory,...

show abstract

Section: Discussionmentioning

confidence: 99%

Hierarchical diffusion models for two-choice response times.

Vandekerckhove

Tuerlinckx

Lee

2011

Psychological Methods

276

324

View full text Add to dashboard Cite

show abstract

“…Klien Entink et al [114] proposed to marry an LLTM approach with the hierarchical modeling approach, as discussed in Section 6.2 [95]. They also employed a full Bayesian framework using Markov Chain Monte Carlo methods to estimate the model.…”

Section: Cognitive Componentsmentioning

confidence: 99%

“…A limitation of the two-step approach is that it fails to model the relationship between responses and response times [114]. Klien Entink et al [114] proposed to marry an LLTM approach with the hierarchical modeling approach, as discussed in Section 6.2 [95].…”

Section: Cognitive Componentsmentioning

confidence: 99%

Use of Response Time for Measuring Cognitive Ability

Kyllonen

2016

J. Intell.

106

View full text Add to dashboard Cite

Abstract:The purpose of this paper is to review some of the key literature on response time as it has played a role in cognitive ability measurement, providing a historical perspective as well as covering current research. We discuss the speed-level distinction, dimensions of speed and level in cognitive abilities frameworks, speed-accuracy tradeoff, approaches to addressing speed-accuracy tradeoff, analysis methods, particularly item response theory-based, response time models from cognitive psychology (ex-Gaussian function, and the diffusion model), and other uses of response time in testing besides ability measurement. We discuss several new methods that can be used to provide greater insight into the speed and level aspects of cognitive ability and speed-accuracy tradeoff decisions. These include item-level time limits, the use of feedback (e.g., CUSUMs), explicit scoring rules that combine speed and accuracy information (e.g., count down timing), and cognitive psychology models. We also review some of the key psychometric advances in modeling speed and level, which combine speed and ability measurement, address speed-accuracy tradeoff, allow for distinctions between response times on items responded to correctly and incorrectly, and integrate psychometrics with information-processing modeling. We suggest that the application of these models and tools is likely to advance both the science and measurement of human abilities for theory and applications.

show abstract

“…Empirical examples of Klein Entink, Fox, and van der Linden (2009) showed that higher-ability test takers tended to work at a slower speed than lower-ability test takers. Klein Entink, Kuhn, Hornke, and Fox (2009), Roberts and Stankov (1999), and van der Linden and Fox (2015) also have reported a negative correlation between ability and speed in their empirical examples.…”

Section: Introductionmentioning

confidence: 99%

Joint Modeling of Ability and Differential Speed Using Responses and Response Times

Fox

Marianti

2016

Multivariate Behavioral Research

Self Cite

View full text Add to dashboard Cite

With computerized testing, it is possible to record both the responses of test takers to test questions (i.e., items) and the amount of time spent by a test taker in responding to each question. Various models have been proposed that take into account both test-taker ability and working speed, with the many models assuming a constant working speed throughout the test. The constant working speed assumption may be inappropriate for various reasons. For example, a test taker may need to adjust the pace due to time mismanagement, or a test taker who started out working too fast may reduce the working speed to improve accuracy. A model is proposed here that allows for variable working speed. An illustration of the model using the Amsterdam Chess Test data is provided.

show abstract

Evaluating cognitive theory: A joint modeling approach using responses and response times.

Cited by 78 publications

References 89 publications

Hierarchical diffusion models for two-choice response times.

Hierarchical diffusion models for two-choice response times.

Use of Response Time for Measuring Cognitive Ability

Joint Modeling of Ability and Differential Speed Using Responses and Response Times

Contact Info

Product

Resources

About