Modeling individual differences in cognition

Abstract

Many evaluations of cognitive models rely on data that have been averaged or aggregated across all experimental subjects, and so fail to consider the possibility of important individual differences between subjects. Other evaluations are done at the single-subject level, and so fail to benefit from the reduction of noise that data averaging or aggregation potentially provides. To overcome these weaknesses, we have developed a general approach to modeling individual differences using families of cognitive models in which different groups of subjects are identified as having different psychological behavior. Separate models with separate parameterizations are applied to each group of subjects, and Bayesian model selection is used to determine the appropriate number of groups. We evaluate this individual differences approach in a simulation study and show that it is superior in terms of the key modeling goals of prediction and understanding. We also provide two practical demonstrations of the approach, one using the ALCOVE model of category learning with data from four previously analyzed category learning experiments, the other using multidimensional scaling representational models with previously analyzed similarity data for colors. In both demonstrations, meaningful individual differences are found and the psychological models are able to account for this variation through interpretable differences in parameterization. The results highlight the potential of extending cognitive models to consider individual differences.

A probabilistic threshold model: Analyzing semantic categorization data with the Rasch model

“…Rather than assuming that all categorizers are alike or that all categorizers are different from one another, one could look for latent groups of similarly performing categorizers. This is the approach taken by, among others, Lee and Webb (2005), Palmeri and Nosofsky (1995), Vanpaemel and Navarro (2007), and Verheyen and Storms (2007). It is straightforward to implement this demand for potential latent classes in the framework of the Probabilistic Threshold Model.…”

Comparing time-accuracy curves: Beyond goodness-of-fit measures

“…The likelihood ratios can be combined across participants in the same way as was described above, but without the constraint that all the participants should be fitted by the same model. For example, Lee and Webb (2005) used the BIC in a similar fashion to identify the number of clusters of participants who showed qualitatively different responses.…”

Assessing individual differences in categorical data

“…Recently, a number of these statistical approaches have begun to appear in the cognitive modeling literature to handle cases where there is participant and/or item parameter heterogeneity. These approaches include finite mixture models (DeCarlo, 2002;Klauer, 2006;Lee & Webb, 2005), Bayesian hierarchical models (e.g., Batchelder & Riefer, 2007;Karabatsos & Batchelder, 2003;Rouder & Lu, 2005; Rouder, Sun, analyses of a violation of the independence assumption; we will say something about the case of continuous data, and we will summarize our general recommendations for analyzing categorical data prior to any cognitive modeling analyses. Throughout the article, we will illustrate our tests with data from cognitive experiments.…”