A general integrative neurocognitive modeling framework to jointly describe EEG and decision-making on single trials

Abstract

Despite advances in techniques for exploring reciprocity in brain-behavior relations, few studies focus on building neurocognitive models that describe both human EEG and behavioral modalities at the single-trial level. Here, we introduce a new integrative joint modeling framework for the simultaneous description of single-trial EEG measures and cognitive modeling parameters of decision making. The new framework can be utilized for the evaluation of research questions as well as the prediction of both data types concurrently. In the introduced joint models, we formalized how single-trial N200 latencies and Centro-parietal positivities (CPP) are predicted by changing single-trial parameters of various drift-diffusion models (DDMs). These models do not have clear closed-form likelihoods and are not easy to fit using Markov chain Monte Carlo (MCMC) methods because nuisance parameters on single trials are shared in both behavior and neural activity. We trained deep neural networks to learn the Bayesian posterior distributions of unobserved neurocognitive parameters based on model simulations. We then used parameter recovery assessment and model misspecification to ascertain how robustly the models’ parameters can be estimated. Moreover, we fit the models to three different real datasets to test their applicability. Our results show that the single-trial integrative models can recover their latent parameters. Finally, we provide some pieces of evidence that single-trial integrative joint models are superior to traditional integrative models. The current single-trial paradigm and the likelihood-free approach for parameter recovery can inspire scientists and modelers to conveniently develop new neuro-cognitive models for other neural measures and to evaluate them appropriately.

Duration discrimination: A diffusion decision modeling approach

“…Implementing such a mechanism into the modeling framework proposed here could lead to an even more complete formalization of all the processes involved in interval timing discrimination and could provide a promising framework for future studies. Moreover, recent advances in joint modeling enable us to incorporate neural data into diffusion models (Ghaderi-Kangavari, Rad, & Nunez, 2022;Turner, Forstmann, & Steyvers, 2019), which could be a fruitful approach to studying brain-behavior relationships during duration discrimination.…”

A tutorial on fitting joint models of M/EEG and behavior to understand cognition

“…Therefore, simulations should be performed to make sure the parameters of interest are recovered when comparing models, as well as understand how model changes affect predictions of multiple data types. For an example of joint modeling for EEG and behavior, see work by Ghaderi-Kangavari, Rad, and Nunez, (2023b).…”

“…One particular promising program is BayesFlow, which finds posterior samples from simulation-based models using invertible neural networks (Radev et al, 2020;Schmitt, Bürkner, Köthe, & Radev, 2022). The first author with colleagues has already used this program with success for Integrative joint modeling of single-trial EEG and behavior during decision-making (Ghaderi-Kangavari et al, 2023b). A similar promising and accessible method is to use neural networks to learn approximate likelihoods that can then be used to find posterior distributions of joint models (Fengler, Govindarajan, Chen, & Frank, 2021;Fengler, Bera, Pedersen, & Frank, 2022).…”

Artificial neural networks for model identification and parameter estimation in computational cognitive models

“…These methods enable automated (or semi-automated) construction of summary statistics, minimizing the effect the choice of summary statistics may have on the accuracy of parameter estimation ( Y. Chen et al, 2020 ; Fearnhead and Prangle, 2012 ; Jiang et al, 2017 ; Lavin et al, 2021 ; Radev, Mertens, et al, 2020 ; Radev, Voss, et al, 2020 ). This innovative approach serves to amortize the computational cost of simulation-based inference, opening new frontiers in terms of scalability and performance ( Boelts et al, 2022 ; Fengler et al, 2021 ; Ghaderi-Kangavari et al, 2023 ; Radev, Mertens, et al, 2020 ; Radev, Voss, et al, 2020 ; Radev et al, 2021 ; Schmitt et al, 2021 ; Sokratous et al, 2023 ).…”