Large-scale analysis of test–retest reliabilities of self-regulation measures

Abstract

The ability to regulate behavior in service of long-term goals is a widely studied psychological construct known as self-regulation. This wide interest is in part due to the putative relations between self-regulation and a range of real-world behaviors. Selfregulation is generally viewed as a trait, and individual differences are quantified using a diverse set of measures, including selfreport surveys and behavioral tasks. Accurate characterization of individual differences requires measurement reliability, a property frequently characterized in self-report surveys, but rarely assessed in behavioral tasks. We remedy this gap by (i) providing a comprehensive literature review on an extensive set of self-regulation measures and (ii) empirically evaluating test-retest reliability of this battery in a new sample. We find that dependent variables (DVs) from self-report surveys of self-regulation have high testretest reliability, while DVs derived from behavioral tasks do not. This holds both in the literature and in our sample, although the test-retest reliability estimates in the literature are highly variable. We confirm that this is due to differences in between-subject variability. We also compare different types of task DVs (e.g., model parameters vs. raw response times) in their suitability as individual difference DVs, finding that certain model parameters are as stable as raw DVs. Our results provide greater psychometric footing for the study of self-regulation and provide guidance for future studies of individual differences in this domain.self-regulation | retest reliability | individual differences These data were previously presented as a poster at

Facet‐level analysis of the relations between personality and laboratory aggression

“…Of note, such concerns about the construct validity of behavioral measures are by no means limited to aggression tasks. Meta‐analytic reviews observe null‐to‐small relations between self‐report and task measures of impulsivity (e.g., Cyders & Coskunpinar, 2011), and recent evidence suggests poor psychometric properties of many functional neuroimaging tasks (Elliott et al, 2019) and behavioral measures of self‐regulation (Enkavi et al, 2019). This criticism, of course, does not excuse or nullify critiques of the CRTT, but it is important to note that the difficulty of integrating self‐report and behavioral measures is a field‐wide dilemma and that the current study found more consistent relations between self‐reported traits and putatively relevant tasks than have been found when using many other laboratory tasks.…”

Temporally-precise disruption of prefrontal cortex informed by the timing of beta bursts impairs human action-stopping

“…Secondly, we have shown the value of using beta bursts to reveal the fine timing of stopping related brain activity (see also Wessel, 2019). Taken together, these methods provide a good basis for testing whether the variation in sub-processes of action-stopping might better relate to real-world self-control and impulsivity than SSRT (Friedman and Miyake, 2004;Lijffijt et al, 2004;McLaughlin et al, 2016;Chowdhury et al, 2017;Enkavi et al, 2019;Skippen et al, 2019).…”

Measuring delay discounting in a crowdsourced sample: An exploratory study

“…We speculate that it may restrict what is being measured to financial risk-taking, and no longer generalize to other manifestations of impulsivity. In fact, a recent study on self-regulation, a related construct to delay discounting, suggests that behavioral measures have lower test-retest reliabilities than survey measures (Enkavi, Eisenberg, Bissett et al, 2019). This finding may reflect the fact that behavioural measures largely assess situation-specific behaviours while self-report measures largely assess traits.…”