Modeling the Covariance Structure of Complex Datasets Using Cognitive Models: An Application to Individual Differences and the Heritability of Cognitive Ability

Abstract

Understanding individual differences in cognitive performance is an important part of understanding how variations in underlying cognitive processes can result in variations in task performance. However, the exploration of individual differences in the components of the decision process-such as cognitive processing speed, response caution, and motor execution speed-in previous research has been limited. Here, we assess the heritability of the components of the decision process, with heritability having been a common aspect of individual differences research within other areas of cognition. Importantly, a limitation of previous work on cognitive heritability is the underlying assumption that variability in response times solely reflects variability in the speed of cognitive processing. This assumption has been problematic in other domains, due to the confounding effects of caution and motor execution speed on observed response times. We extend a cognitive model of decision-making to account for relatedness structure in a twin study paradigm. This approach can separately quantify different contributions to the heritability of response time. Using data from the Human Connectome Project, we find strong evidence for the heritability of response caution, and more ambiguous evidence for the heritability of cognitive processing speed and motor execution speed. Our study suggests that the assumption made in previous studies-that the heritability of cognitive ability is based on cognitive processing speed-may be incorrect. More generally, our methodology provides a useful avenue for future research in complex data that aims to analyze cognitive traits across different sources of related data, whether the relation is between people, tasks, experimental phases, or methods of measurement.

Evidence Accumulation Models: Current Limitations and Future Directions

“…Interestingly, Ratcliff et al (2001) found that in several tasks younger and older adults had very similar drift rates, and the slower performance of older adults was actually the result of greater caution in responding (i.e., higher decision threshold) and slower perceptual/motor processes (i.e., higher non-decision time), showing evidence against the cognitive slowdown theory through the EAM framework. EAMs have been able to answer similarly posed questions in a range of different paradigms, such as letter identification (Ratcliff & Rouder, 2000), lexical decision-making (Wagenmakers, Ratcliff, Gomez, & McKoon, 2008), sentence comprehension (Lerche, Christmann, & Voss, 2019), genetic heritability (Evans, Steyvers, & Brown, 2018), intelligence testing (Ratcliff, Thapar, & McKoon, 2010), recognition memory (Ratcliff, 1978), personality (Evans, Rae, Bushmakin, Rubin, & Brown, 2017), early life adversity (Knowles, Evans, & Burke, 2019), and performance optimality (Starns & Ratcliff, 2012;Evans, Bennett, & Brown, 2018).…”

“…Although these applications can provide interesting information about the task and provide theoretically relevant inferences about cognitive processes, they represent a small subset of the poten-The Quantitative Methods for Psychology tial questions that EAMs could be used to answer; something that we discuss in more detail within our "Future Directions for EAMs as Measurement Tools" section. Although recent research has attempted extend EAM applications to new realms, such as joint modelling approaches that attempt to link multiple sources of data (Turner et al, 2013;Evans, Rae, et al, 2017;Turner, Rodriguez, Norcia, McClure, & Steyvers, 2016;Evans, Steyvers, & Brown, 2018;Turner, Van Maanen, & Forstmann, 2015;Knowles et al, 2019;Turner, Wang, & Merkle, 2017;Krajbich, Armel, & Rangel, 2010), these applications appear to be exception, rather than the rule. This suggests that the EAM framework may encourage researchers to answer restrictive questions, with EAM applications mostly revolving around the question "does drift rate or threshold vary between these conditions or groups?…”

“…Joint modelling techniques vary greatly from study to study, such as creating a full covariance matrix across all parameters from two models of two different types of data (Turner et al, 2013), estimating the correlation between model parameters and a different source of data (Knowles et al, 2019), estimating the correlation between the same parameters across different people or experimental conditions (Evans, Steyvers, & Brown, 2018), or restricting a specific parameter of a model to be a function of another source of data (Evans, Rae, et al, 2017). The types of data also varies greatly from study to study, with previous research having combined EAMs with neural data (Turner et al, 2013;Turner et al, 2016;Turner et al, 2015;Turner et al, 2017), personality data (Evans, Rae, et al, 2017), eye-tracking data (Krajbich et al, 2010), genetics data (Evans, Steyvers, & Brown, 2018), EMG data (Servant et al, 2016) and developmental data (Knowles et al, 2019). Importantly, these methods can be combined with psychophysiological data, such as neural recordings (e.g., single cell recordings, electroencephalography [EEG], functional magnetic resonance imaging [fMRI]) and motor recordings (e.g., EMG), which would provide further insight into how these data that reflect other sub-processes relate to the latent parameters within EAMs.…”

Robust Standards in Cognitive Science

“…Model application consists of studies where an existing cognitive model, which is assumed to provide an adequate representation of the underlying cognitive process, is applied to empirical data to provide insight into how the cognitive process operates in that paradigm (e.g., Weigard & Huang-Pollock, 2017;Ratcliff et al, 2001;Janczyk & Providing a clear and transparent documentation of the model exploration process Discussing existing theoretical justification for model components and functional form Distinguishing between theory-driven development and data-driven development Deciding which components of the model are core and which are ancillary Deciding upon the purpose of the model (e.g., tool for application, formalization of theory, both) Deciding upon evaluation criteria that will drive the model development (e.g., data trends, parameter identifiability) Lerche et al, 2019;Wagenmakers et al, 2008;Ratcliff & Rouder, 2000;Evans, Bennett, & Brown, 2018;Evans, Steyvers, & Brown, 2018). These applications often involve experimental studies with different groups and/or conditions, with researchers interested in how the cognitive process changes across these factors, measured by changes in the values of the model parameters.…”

Thermodynamic integration via differential evolution: A method for approximating marginal likelihoods

“…This is especially true when using population Markov chain Monte Carlo (MCMC) techniques, such as differential evolution MCMC (DE-MCMC;Ter Braak, 2006;Turner et al, 2013), to sample from the power posteriors as they require several chains for each power posterior. The DE-MCMC approach has become popular due to its ability to efficiently sample from models with correlated parameters (see Evans, Rae, Bushmakin, Rubin, & Brown, 2017;Evans, Steyvers, & Brown, 2018 for some applications) by using a set of interacting chains, usually 2-3 times number of individual-level parameters. Thus, DE-MCMC can be computationally burdensome when used in the context of TI, especially as the number of individual-level parameters grows.…”