Cognitive model exploration and optimization: a new challenge for computational science

Moore, Jr. L. Richard

doi:10.1007/s10588-011-9092-8

Cited by 13 publications

(9 citation statements)

References 13 publications

(7 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One issue that has been raised is the fact that manual parameter tuning is still commonly used instead of automatic parameter inference methods (Lane & Gobet, 2013;Raymond, Fornberg, Buck-Gengler, Healy, & Bourne, 2008;Said, Engelhart, Kirches, K€ orkel, & Holt, 2016). Another issue has been the fact that the most basic automatic inference methods might either not be efficient enough or be unable to visualize the model fit over the parameter space (Gluck, Stanley, Moore, Reitter, & Halbr€ ugge, 2010;Moore, 2011).…”

Section: Introductionmentioning

confidence: 99%

Parameter Inference for Computational Cognitive Models with Approximate Bayesian Computation

et al. 2019

View full text Add to dashboard Cite

This paper addresses a common challenge with computational cognitive models: identifying parameter values that are both theoretically plausible and generate predictions that match well with empirical data. While computational models can offer deep explanations of cognition, they are computationally complex and often out of reach of traditional parameter fitting methods. Weak methodology may lead to premature rejection of valid models or to acceptance of models that might otherwise be falsified. Mathematically robust fitting methods are, therefore, essential to the progress of computational modeling in cognitive science. In this article, we investigate the capability and role of modern fitting methods—including Bayesian optimization and approximate Bayesian computation—and contrast them to some more commonly used methods: grid search and Nelder–Mead optimization. Our investigation consists of a reanalysis of the fitting of two previous computational models: an Adaptive Control of Thought—Rational model of skill acquisition and a computational rationality model of visual search. The results contrast the efficiency and informativeness of the methods. A key advantage of the Bayesian methods is the ability to estimate the uncertainty of fitted parameter values. We conclude that approximate Bayesian computation is (a) efficient, (b) informative, and (c) offers a path to reproducible results.

show abstract

Section: Introductionmentioning

confidence: 99%

Parameter Inference for Computational Cognitive Models with Approximate Bayesian Computation

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Developing techniques for efficient parameter space exploration and parameter estimation is still a relatively new research area in cognitive modeling, and only a few systematic approaches have been described in the literature to date, e.g. [ 9 – 13 ]. Systematic exploration of a model’s parameter space is often desirable, but quickly runs into difficulties, as processing time increases exponentially with the number of parameters and the resolution of analysis ( curse of dimensionality ).…”

Section: Introductionmentioning

confidence: 99%

“…Another possibility is to improve the efficiency of search algorithms. One approach is to sample the search space selectively, for example using adaptive mesh refinement or regression trees [ 9 , 13 ], where regions of the search space with high-information content are sampled more densely. This strategy allows to preserve most of the information relevant for modeling purposes, while reducing the number of samples required.…”

Section: Introductionmentioning

confidence: 99%

Applying Mathematical Optimization Methods to an ACT-R Instance-Based Learning Model

et al. 2016

View full text Add to dashboard Cite

Computational models of cognition provide an interface to connect advanced mathematical tools and methods to empirically supported theories of behavior in psychology, cognitive science, and neuroscience. In this article, we consider a computational model of instance-based learning, implemented in the ACT-R cognitive architecture. We propose an approach for obtaining mathematical reformulations of such cognitive models that improve their computational tractability. For the well-established Sugar Factory dynamic decision making task, we conduct a simulation study to analyze central model parameters. We show how mathematical optimization techniques can be applied to efficiently identify optimal parameter values with respect to different optimization goals. Beyond these methodological contributions, our analysis reveals the sensitivity of this particular task with respect to initial settings and yields new insights into how average human performance deviates from potential optimal performance. We conclude by discussing possible extensions of our approach as well as future steps towards applying more powerful derivative-based optimization methods.

show abstract

“…However, most theories in psychology and cognitive science have a number of parameters, and this number can sometimes be considerable. In spite of some efforts in this direction (Kase, Ritter, and Schoelles, 2008;Moore Jr, 2011;Ritter, 1991;Tor and Ritter, 2004), few modellers (outside mathematical psychology) routinely use formal, automated techniques to optimise the parameters and/or the structural components of their models. The fact that most cognitive models are not optimised poses important problems for building and evaluating theories.…”

Section: Introductionmentioning

confidence: 99%

Evolving Non-Dominated Parameter Sets for Computational Models from Multiple Experiments

Lane

Gobet

2013

Journal of Artificial General Intelligence

View full text Add to dashboard Cite

?? Peter C. R. Lane, Fernand Gobet. This article is distributed under the terms of the Creative Commons Attribution Non-Commercial License, which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited. (CC BY-NC-ND 3.0)Creating robust, reproducible and optimal computational models is a key challenge for theorists in many sciences. Psychology and cognitive science face particular challenges as large amounts of data are collected and many models are not amenable to analytical techniques for calculating parameter sets. Particular problems are to locate the full range of acceptable model parameters for a given dataset, and to confirm the consistency of model parameters across different datasets. Resolving these problems will provide a better understanding of the behaviour of computational models, and so support the development of general and robust models. In this article, we address these problems using evolutionary algorithms to develop parameters for computational models against multiple sets of experimental data; in particular, we propose the ???speciated non-dominated sorting genetic algorithm??? for evolving models in several theories. We discuss the problem of developing a model of categorisation using twenty-nine sets of data and models drawn from four different theories. We find that the evolutionary algorithms generate high quality models, adapted to provide a good fit to all available data

show abstract

Cognitive model exploration and optimization: a new challenge for computational science

Cited by 13 publications

References 13 publications

Parameter Inference for Computational Cognitive Models with Approximate Bayesian Computation

Parameter Inference for Computational Cognitive Models with Approximate Bayesian Computation

Applying Mathematical Optimization Methods to an ACT-R Instance-Based Learning Model

Evolving Non-Dominated Parameter Sets for Computational Models from Multiple Experiments

Contact Info

Product

Resources

About