Interference scores have inadequate concurrent and convergent validity: Should we stop using the flanker, Simon, and spatial Stroop tasks?

Abstract

Background: Two-hundred one college undergraduates completed four nonverbal interference tasks (Simon, spatial Stroop, vertical Stroop, and flanker) and trait scales of self-control and impulsivity. Regression analyses tested 11 predictors of the composite interference scores derived from three of the four tasks and each task separately. The purpose of the study was to examine the relationships between laboratory measures of self-control, self-report measures, and the degree to which control might be related to extensive experience in activities that logically require self-control. Results: Fluid intelligence and sex were significant predictors of the composite measure, but bilingualism, music training, video gaming, mindfulness/meditation, self-control, impulsivity, SES, and physical exercise were not. Conclusions: Common laboratory measures of inhibitory control do not correlate with self-reported measures of selfcontrol or impulsivity and consequently appear to be measuring different constructs. Bilingualism, mindfulness/ meditation, playing action video games, and music training or performance provide weak and inconsistent improvements to laboratory measures of interference control. Flanker, Simon, and spatial Stroop effects should not be used or interpreted as measures of domain-general inhibitory control.

Autism Traits Predict Self-reported Executive Functioning Deficits in Everyday Life and an Aversion to Exercise

“…Given that this is a set of 25 correlations and these bivariate correlations were not corrected for multiple tests, one might argue that there is very little evidence that self-control scales and laboratory measures of EF are measuring the same construct. This is consistent with the results and conclusions of Paap et al (2020a), Toplak et al (2013), and Duckworth and Kern (2011).…”

Developmental trajectories of metacognitive processing and executive function from childhood to older age

“…The development and decline of inhibitory control and monitoring followed the same trajectory in both monolingual and multilingual speakers. However, although the Simon test is widely employed as a measure of inhibition, we also acknowledge that reported correlations of performance across tests designed to measure inhibition are frequently low, and that this observation has led authors to question the convergent validity of the term, and therefore its usefulness in the literature (e.g., Paap et al, 2020; Rey-Mermet et al, 2018).…”

Do All Switches Cost the Same? Reliability of Language Switching and Mixing Costs

“…Many other cognitive tasks that use difference scores to measure a specific cognitive component also produce robust effects at the group level, but fail to show reliability as a measure of individual differences. For example, the commonly used difference scores reflecting the ability to resist interference in the Stroop and flanker tasks, show low reliability ( Paap & Sawi, 2016 ; Paap, Anders-Jefferson, Zimiga, Mason, & Mikulinsky, 2020 ; Siegrist, 1997 ; Von Bastian et al, 2016 ). These measures are also only weakly correlated with each other ( Prior et al, 2017 ; Rey-Mermet, Gade, & Oberauer, 2018 ; Rouder & Haaf, 2019 ) even though they are thought to rely on similar processes ( Draheim et al, 2020 ).…”