Inclusion Bayes Factors for Mixed Hierarchical Diffusion Decision Models

Abstract

Cognitive models provide a substantively meaningful quantitative description of latent cognitive processes. The quantitative formulation of these models supports cumulative theory building and enables strong empirical tests. However, the non-linearity of these models and pervasive correlations among model parameters pose special challenges when applying cognitive models to data. Firstly, estimating cognitive models typically requires large hierarchical data sets that need to be accommodated by an appropriate statistical structure within the model. Secondly, statistical inference needs to appropriately account for model uncertainty to avoid overconfidence and biased parameter estimates. In the present work we show how these challenges can be addressed through a combination of Bayesian hierarchical modelling and Bayesian model averaging. To illustrate these techniques, we apply the popular diffusion decision model to data from a collaborative selective influence study.

Uncovering the cognitive mechanisms underlying the gaze cueing effect

“…After assessing the weighted probabilities of each respective model, we then assessed the relative influence of each parameter of interest (non-decision time, starting point, and drift rate). This was achieved by combining the relative probability, according to BIC, of models that contained the same assumption about each respective parameter (see Boehm et al, 2021 , note that only the BIC parameter inclusion probabilities are presented in this manuscript as the qualitative trends were equivalent for AIC, but the AIC inclusion probabilities can be found in Supplementary Material 6 ). We then compared them with models that do not make the same assumption in order to identify the relative importance of that assumption.…”

EMC2: An R Package for cognitive models of choice

“…The third, marginal deviance (MD), the basis of the gold standard Bayes factor (Kass & Raftery, 1995), requires integration over the prior. EMC2 uses an implementation of Warp-III bridge sampling (Gronau et al, 2020(Gronau et al, , 2017 to achieve likelihood integration that is much more efficient than those previously used with cognitive models (Boehm et al, 2024;Gronau et al, 2020). The computation for the two models took less than 20 seconds.…”

“…Different choices might be of interest, such as setting different priors, determining if the DDM could be made more competitive by more flexible boundary and rate bias effects, and perhaps exploring analogous effects in the LBA. We also note that users may prefer to use model-averaging approaches rather than selecting a "best" model (Boehm et al, 2024), with all of these possibilities being relatively straightforward to implement in EMC2.…”

Uncovering the Cognitive Mechanisms Underlying the Gaze Cueing Effect

“…After assessing the weighted probabilities of each respective model, we then assessed the relative influence of each parameter of interest (non-decision time, starting point, and drift rate). This was achieved by combining the relative probability, according to BIC, of models that contained the same assumption about each respective parameter (see Boehm et al, 2021, note that only the BIC parameter inclusion probabilities are presented in this manuscript as the qualitative trends were equivalent for AIC, but the AIC inclusion probabilities can be found in supplementary material 8 ). We then compared them to models that do not make the same assumption in THE COGNITIVE MECHANISMS UNDERLYING GAZE CUEING 22 order to identify the relative importance of that assumption.…”