Artificial neural networks for model identification and parameter estimation in computational cognitive models

Abstract

Computational cognitive models have been used extensively to formalize cognitive processes. Model parameters offer a simple way to quantify individual differences in how humans process information. Similarly, model comparison allows researchers to identify which theories, embedded in different models, provide the best accounts of the data. Cognitive modeling uses statistical tools to quantitatively relate models to data that often rely on computing/estimating the likelihood of the data under the model. However, this likelihood is computationally intractable for a substantial number of models. These relevant models may embody reasonable theories of cognition, but are often under-explored due to the lack of tools required to relate them to data. We propose to fill this gap using artificial neural networks (ANNs) to map data directly onto model identity and parameters, bypassing the likelihood estimation. Our results show that we can adequately perform both parameter estimation and model identification using our new ANN approach, including for models that cannot be fit using traditional likelihood-based methods. Our new ANN approach will greatly broaden the class of cognitive models researchers can quantitatively consider.

Active reinforcement learning versus action bias and hysteresis: control with a mixture of experts and nonexperts

“…Following the trends of artificial neural networks, deep learning [247][248][249][250][251], and deep RL [252][253][254][255][256][257][258][259][260], recent approaches to cognitive modeling have begun to utilize machine learning via the architecture of a recurrent neural network (RNN) [261][262][263]-such as with a long short-term memory (LSTM) unit [264] or a simpler gated recurrent unit (GRU) [265]-in an attempt to understand core computations for learning (i.e., beyond just nonlinear function approximation for state representation) [266][267][268][269][270][271][272][273][274][275][276][277][278][279]. Whereas such efforts pursue a data-centric approach leveraging predictive power as opposed to the present theory-centric approach PLOS COMPUTATIONAL BIOLOGY leveraging explanatory power, it is the latter that has so far proven more effective for inference about empirical behavior (but see [87,280]).…”

Latent Learning Progress Drives Autonomous Goal Selection in Human Reinforcement Learning

“…The options model was unfittable through HBI methods due to intractable likelihood [101]. To approximate the learning parameters for each participant (β learn , α learn , and ϕ), we fit a simpler reinforcement learning model (without options), this time through maximum-likelihood methods and only on action selection date, and used the resulting best-fit learning parameters for simulations.…”

“…These results suggest at least some participants generated insights for learning by recognizing hierarchical relationships in the learning environment, while not necessarily using this information directly to decide which goals to select. Since the options model was not fittable through HBI methods, quantitative confirmation of these results awaits formal model comparison through more advanced model fitting methods [101].…”