Exploration for Understanding in Cognitive Modeling

Abstract

The cognitive modeling and artificial general intelligence research communities may reap greater scientific return on research investmentsmay achieve an improved understanding of architectures and modelsif there is more emphasis on systematic sensitivity and necessity analyses during model development, evaluation, and comparison. We demonstrate this methodological prescription with two of the models submitted for the Dynamic Stocks and Flows (DSF) Model Comparison Challenge, exploring the complex interactions among architectural mechanisms, knowledge-level strategy variants, and task conditions. To cope with the computational demands of these analyses we use a predictive analytics approach similar to regression trees, combined with parallelization on high performance computing clusters, to enable large scale, simultaneous search and exploration.

Understanding multitasking through parallelized strategy exploration and individualized cognitive modeling

“…The specific cluster used here is part of the University of Oregon's Applied Computational Instrument for Scientific Synthesis. Though previous research [4] used a computer cluster to explore model parameters, our system pushes the boundary further by using a cluster to also explore task strategies. Figure 5 shows the components of the PCM system and illustrates how it generates and parallelizes multiple models at once.…”

Cognitive model exploration and optimization: a new challenge for computational science

“…Model strategies, however, can be considered a categorical variable, which presents a challenge for both regression tree and AMR approaches. One approach to evaluating multiple strategies is to conduct a separate search for each strategy parameter space and compare the results, as was demonstrated in Gluck et al (2010). It would be better, however, to conduct a single search across all the strategies because a direct comparison during the search process could result in strategy down-selections that improve efficiency.…”

“…Furthermore, recognizing the necessity for qualitatively understanding the parameter space, I also developed accompanying visualization software that operates in real time as the space is constructed. The visualization software is called Hurricane, while the intelligent search software is called Cell Gluck et al 2010).…”

12th Biannual conference of the German cognitive science society (Gesellschaft für Kognitionswissenschaft)

“…While an acceptable fit of the model is necessary to support the activation spreading hypothesis, it is not sufficient to prove it. By manipulating the amount of activation spreading, we can perform a sensitivity and necessity analysis that provides additional insight about the consequences of our theoretical assumptions (Gluck et al 2010). Average coefficients from a total of 400 model runs are displayed in Fig.…”