Value-free random exploration is linked to impulsivity

Abstract

Deciding whether to forgo a good choice in favour of exploring a potentially more rewarding alternative is one of the most challenging arbitrations both in human reasoning and in artificial intelligence. Humans show substantial variability in their exploration, and theoretical (but only limited empirical) work has suggested that excessive exploration is a critical mechanism underlying the psychiatric dimension of impulsivity. In this registered report, we put these theories to test using large online samples, dimensional analyses, and computational modelling. Capitalising on recent advances in disentangling distinct human exploration strategies, we not only demonstrate that impulsivity is associated with a specific form of exploration—value-free random exploration—but also explore links between exploration and other psychiatric dimensions.

Developmental changes resemble stochastic optimization

“…2). We treat age as a continuous variable when possible, but also discretize participants into 7 similarly sized age groups (n ∈ [30,50]). These behavioral results reveal clear age-related trends in learning and exploration captured by our task.…”

“…In each round, a new environment was chosen without replacement from the list of environments. To prevent participants from knowing when they found the highest reward, a different maximum range was sampled from a uniform distribution ∼ U (30,40) for each round and all reward values were rescaled accordingly. The rescaled rewards were then shifted by +5 to avoid reward observations below 0.…”

“…Perhaps the most direct interpretation is to apply "cooling off" to the single dimension of random temperature, controlling the amount of decision noise when selecting actions or sampling hypotheses 6,16,27 . Evidence from experimental studies suggest that young children are harder to predict than adults 17,28 , implying greater stochasticity, which is amplified in neurodevelopmental disorders such as attention deficit hyperactivity disorder (ADHD) 29 and impulsivity 30 . However, this interpretation is only part of the story, since developmental differences in choice variability can be traced to changes affecting multiple aspects of learning and choice behavior.…”

From Puzzle to Progress: How engaging with Neurodiversity can improve Cognitive Science

“…Neurodiversity thus presents an opportunity for cognitive science to build fundamentally new models of cognition. These could involve replacing single-agent reinforcement learning models with populations pursuing heterogenous strategies (related to neurodivergence by Dubois & Hauser, 2022;Yechiam et al, 2010). They could challenge the concept "theory of mind" (Williams, 2004) by extending findings that ToM is not monolithic (Ahmed & Miller, 2011;Schaafsma et al, 2015;Warnell & Redcay, 2019) and showing how different neurotypes use different strategies in doing ToM tasks.…”

“…Diversity is especially important for scientific discovery: identifying questions, framing problems, or uncovering insights (Sulik et al, 2021b). Neurodivergent people explore in different ways than neurotypicals (Dubois & Hauser, 2022;Yechiam et al, 2010) and exploration is part of discovery (Devezer et al, 2019). A variety of strategies-incorporating research led by both neurodivergent and neurotypical peoplecould lead to better long-term advances.…”