Speed–Accuracy Trade-Off? Not So Fast: Marginal Changes in Speed Have Inconsistent Relationships With Accuracy in Real-World Settings

Abstract

The speed–accuracy trade-off (SAT) suggests that time constraints reduce response accuracy. Its relevance in observational settings—where response time (RT) may not be constrained but respondent speed may still vary—is unclear. Using 29 data sets containing data from cognitive tasks, we use a flexible method for identification of the SAT (which we test in extensive simulation studies) to probe whether the SAT holds. We find inconsistent relationships between time and accuracy; marginal increases in time use for an individual do not necessarily predict increases in accuracy. Additionally, the speed–accuracy relationship may depend on the underlying difficulty of the interaction. We also consider the analysis of items and individuals; of particular interest is the observation that respondents who exhibit more within-person variation in response speed are typically of lower ability. We further find that RT is typically a weak predictor of response accuracy. Our findings document a range of empirical phenomena that should inform future modeling of RTs collected in observational settings.

Event-Based Prospective Memory in Young Adults: The Role of Gender Differences

“…This is illustrated by recent research, which found that accuracy cannot always be predicted by reaction time. Therefore, a longer time can indicate either increased or reduced accuracy (32). This issue may be caused by characteristics that vary between male and female traits.…”

The Item Response Warehouse (IRW), Version 4.0

“…As a illustration of an alternative approach, we conducted a study of the speed-accuracy tradeoff's relevance in observational data using a data-rich paradigm [2]. Our approach suggested that a wide variety of conditional accuracy functions (CAFs) may be observed in empirical settings.…”

Differences in time usage as a competing hypothesis for observed group differences in accuracy with an application to observed gender differences in PISA data

“…Recent work [25,26] has developed DIF methods to detect time use differences between groups, and has shown them to be related to response accuracy in tests like PISA. The relationship between time use and accuracy has been shown to exist even after controlling for item and person effects, and can also take complicated functional forms [27,28].…”

“…These studies typically use experimental interventions designed to force respondents to respond at different speeds. However, more recent work focuses on identification of the CAF in observational settings [34,27] in an attempt to ask whether idiosyncratic within-person variation in speed is associated with variation in accuracy. We build on this approach here.…”

“…For example, van der Linden's hierarchical model suggests no change in accuracy as a function of time use [35]; i.e., a flat CAF. Empirically, a range of relationships seem possible [27,28], including downward-sloping CAFs or curvilinear CAFs where accuracy first increases with more time and then decreases, possibly due to disengagement or confusion. Second, conventional ability estimates that are based purely on observed accuracy would be unable to distinguish between the speed-and capacity-driven differences in accuracy that we discuss above.…”