Parameter Inference for Computational Cognitive Models with Approximate Bayesian Computation

Abstract

This paper addresses a common challenge with computational cognitive models: identifying parameter values that are both theoretically plausible and generate predictions that match well with empirical data. While computational models can offer deep explanations of cognition, they are computationally complex and often out of reach of traditional parameter fitting methods. Weak methodology may lead to premature rejection of valid models or to acceptance of models that might otherwise be falsified. Mathematically robust fitting methods are, therefore, essential to the progress of computational modeling in cognitive science. In this article, we investigate the capability and role of modern fitting methods—including Bayesian optimization and approximate Bayesian computation—and contrast them to some more commonly used methods: grid search and Nelder–Mead optimization. Our investigation consists of a reanalysis of the fitting of two previous computational models: an Adaptive Control of Thought—Rational model of skill acquisition and a computational rationality model of visual search. The results contrast the efficiency and informativeness of the methods. A key advantage of the Bayesian methods is the ability to estimate the uncertainty of fitted parameter values. We conclude that approximate Bayesian computation is (a) efficient, (b) informative, and (c) offers a path to reproducible results.

Computational Models of Retrieval Processes in Sentence Processing

“…The participant-level estimates of the interference effect were computed from a hierarchical Bayesian model [86]. The average facilitatory interference effect is shown as a 95% credible interval (labeled Observed effect), and the range of predicted values from the activation model are also shown as a 95% credible interval (labeled Model prediction); the model predictions are computed using Approximate Bayesian Computation [85,87,88]. We see the predicted facilitatory interference effect even at the individual participant level, but different participants show varying magnitudes.…”

Machine learning and artificial neural network accelerated computational discoveries in materials science

“…As abovementioned, each algorithm may have several hyperparameters. To help find the best parameters for a particular model, the grid search method can be applied …”

“…To help find the best parameters for a particular model, the grid search method can be applied. 56 In supervised regression problems, accessible performance measurements are the root mean square error (RMSE), mean square error (MSE), and mean absolute errors (MAEs). The MSE is expressed as…”

Interference patterns in subject-verb agreement and reflexives revisited: A large-sample study

“…Previous papers using the LV05 model (most recently, Engelmann et al, 2019) have used grid search to obtain point value estimates of parameters when fitting the model to data. ABC represents a superior approach (Kangasrääsiö, Jokinen, Oulasvirta, Howes, & Kaski, 2019) because we can obtain a posterior distribution of the parameter (or of multiple parameters) of interest. The posterior distribution of the parameter allows us to incorporate uncertainty about the true value of the parameter in our model predictions, instead of using estimated point values for the parameter.…”