Same model, different conclusions: An identifiability issue in the linear ballistic accumulator model of decision-making

Abstract

Evidence accumulation models (EAMs) – the dominant modelling framework for speeded decision-making – have become an important tool for model application. Model application involves using specific model to estimate parameter values that relate to different components of the cognitive process, and how these values differ over experimental conditions and/or between groups of participants. In this context, researchers are often agnostic to the specific theoretical assumptions made by different EAM variants, and simply desire a model that will provide them with an accurate measurement of the parameters that they are interested in. However, recent research has suggested that the two most commonly applied EAMs – the diffusion model and the linear ballistic accumulator (LBA) – come to fundamentally different conclusions when applied to the same empirical data. The current study provides an in-depth assessment of the measurement properties of the two models, as well as the mapping between, using two large scale simulation studies and a reanalysis of Evans (2020a). Importantly, the findings indicate that there is a major identifiability issue within the standard LBA, where differences in decision threshold between conditions are practically unidentifiable, which appears to be caused by a tradeoff between the threshold parameter and the overall drift rate across the different accumulators. While this issue can be remedied by placing some constraint on the overall drift rate across the different accumulators – such as constraining the average drift rate or the drift rate of one accumulator to have the same value in each condition – these constraints can qualitatively change the conclusions of the LBA regarding other constructs, such as non-decision time. Furthermore, all LBA variants considered in the current study still provide qualitatively different conclusions to the diffusion model. Importantly, the current findings suggest that researchers should not use the unconstrained version of the LBA for model application, and bring into question the conclusions of previous studies using the unconstrained LBA.

A model-based approach to disentangling facilitation and interference effects in conflict tasks

“…To select the generating sets of parameter values, we used a Latin hypercube sampling design, which provides the most efficient method for sampling from a multi-dimensional continuous space (McKay, Beckman, & Conover, 1979;Evans, Trueblood, & Holmes, 2020;Evans, 2020a). We sampled 3,000 parameter sets from the hypercube in the first simulation study, and 10,000 in the other simulation studies due to the wider range of parameters, generating a single data set from each parameter set.…”

Hidden Markov Models of Evidence Accumulation in Speeded Decision Tasks

“…Although the LBA became a popular choice for analyzing response times and accuracy, more recently evidence has surfaced suggesting practical identifiability issues of the standard LBA model -especially when trying to quantify differences in parameters such as decision boundary or drift rates between experimental conditions (N. Evans, 2020). Given that HMMs can be viewed as way to quantify differences between "conditions" (states) which themselves need to be inferred from the data, (lack of) identifiability of the standard LBA is a concern.…”

“…For example, a relatively well established set of constraints is to ensure that the average drift rates across accumulators are equal to some constant value (e.g. 1 Donkin, Brown, Heathcote, & P r e p r i n t Wagenmakers, 2011;N. Evans, 2020;Visser & Poessé, 2017)).…”

“…Such constraints may be accompanied by implementing equality constraints on parameters such as the upper bound of the starting point or the standard deviation of the drift rates. In the context of different conditions, even more stringent (equality) constraints are possible, such as equating parameters (such as drift rate for the "error" response) across conditions (N. Evans, 2020).…”

“…The only successful applications of HMMs in these tasks is in combination with models that cannot capture possible residual dependencies, usually log-normal models or shifted Wald models for response times (Dutilh et al, 2010;Molenaar et al, 2016;Timmers, 2019). Yet, even the supposedly simplest complete model of response times and accuracy -the Linear Ballistic Accumulation model (LBA, S. D. Brown & Heathcote, 2008) -has proven to be difficult to combine with an HMM structure or even as a simple independent mixture (Veldkamp, 2020); this may not come as a surprise considering the general identifiability issues of the standard LBA model (N. Evans, 2020).…”