Development of directed and random exploration in children

Meder, Björn; Wu, Charley M.; Schulz, Eric; Ruggeri, Azzurra

doi:10.1111/desc.13095

Cited by 63 publications

(77 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…higher payout). This is in line with the conceptualization of random exploration as an inferior exploration strategy (Meder et al, 2021), which requires only little cognitive resources and is implemented in a simpler fashion into cognitive and neural processes by using neural or environmental noise to randomize choice (Zajkowski et al, 2017). Directed exploration, on the other hand, as reflected in the exploration bonus parameter 𝜑, showed an inverted-U-shaped association with task performance.…”

Section: Implications For Empirical Researchsupporting

confidence: 81%

“…Random exploration has low computational costs but does not lead to optimal performance. Directed exploration is a more elaborate strategy involving more computational costs and leading to better performance (Meder et al, 2021;Wilson et al, 2014). Another phenomenon occurring in explore-or-exploit situations is perseveration, i.e.…”

Section: Introductionmentioning

confidence: 99%

“…Bandit tasks offer a realistic and widely applicable operationalization of the exploration-exploitation trade-off, for situations in which the properties of the environment are independent of the agent's behavior. There are various specialized versions of the bandit task, including the horizon task (Wilson et al, 2014), the leapfrog task (Blanco et al, 2013;Knox et al, 2011), and a version of the bandit task designed for testing on children (Meder et al, 2021). One task version widely used in cognitive neuroscience is the four-armed restless bandit task (Daw et al, 2006).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Parameter and model recovery of reinforcement learning models for restless bandit problems

Danwitz

Mathar

Smith

et al. 2021

Preprint

View full text Add to dashboard Cite

Multi-armed restless bandit tasks are regularly applied in psychology and cognitive neuroscience to assess exploration and exploitation behavior in structured environments. These models are also readily applied to examine effects of (virtual) brain lesions on performance, and to infer neurocomputational mechanisms using neuroimaging or pharmacological approaches. However, to infer individual, psychologically meaningful parameters from such data, computational cognitive modeling is typically applied. Recent studies indicate that softmax (SM) decision rule models that include a representation of environmental dynamics (e.g. the Kalman Filter) and additional parameters for modeling exploration and perseveration (Kalman SMEP) fit human bandit task data better than competing models. Parameter and model recovery are two central requirements for computational models: parameter recovery refers to the ability to recover true data-generating parameters; model recovery refers to the ability to correctly identify the true data generating model using model comparison techniques. Here we comprehensively examined parameter and model recovery of the Kalman SMEP model as well as nested model versions, i.e. models without the additional parameters, using simulation and Bayesian inference. Parameter recovery improved with increasing trial numbers, from around .8 for 100 trials to around .93 for 300 trials. Model recovery analyses likewise confirmed acceptable recovery of the Kalman SMEP model. Model recovery was lower for nested Kalman filter models as well as delta rule models with fixed learning rates.Exploratory analyses examined associations of model parameters with model-free performance metrics. Random exploration, captured by the inverse softmax temperature, was associated with lower accuracy and more switches. For the exploration bonus parameter modeling directed exploration, we confirmed an inverse-U-shaped association with accuracy, such that both an excess and a lack of directed exploration reduced accuracy. Taken together, these analyses underline that the Kalman SMEP model fulfills basic requirements of a cognitive model.

show abstract

Section: Implications For Empirical Researchsupporting

confidence: 81%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Parameter and model recovery of reinforcement learning models for restless bandit problems

Danwitz

Mathar

Smith

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…They also provide the foundation of active learning strategies used to adaptively sample high-dimensional search spaces, for instance in material sciences (Lookman et al, 2019), drug discovery (Murphy, 2011), or learning of protein interactions (Mohamed et al, 2010). How humans search large decision spaces has also been addressed in recent psychological research (Meder et al, 2021;Schulz et al, 2019;Wu et al, 2018).…”

Section: What Makes a Good Query?mentioning

confidence: 99%

What makes a good query? Prospects for a comprehensive theory of human information acquisition

Meder¹,

Crupi²,

Nelson³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Searching for information in a goal-directed manner is central for learning, diagnosis, and prediction. Children continuously ask questions to learn new concepts, doctors do medical tests to diagnose their patients, and scientists perform experiments to test their theories. But what makes a good question? What principles govern human information acquisition and how do people decide which query to conduct to achieve their goals? What challenges need to be met to advance theory and psychology of human inquiry? Addressing these issues, we introduce the conceptual and mathematical ideas underlying different models of the value of information, what purpose these models serve in psychological research, and how they can be integrated in a unified formal framework. We also discuss the conflict between short- and long-term efficiency of prominent methods for query selection, and the resulting normative and methodological implications for studying human sequential search. A final point of discussion concerns the relations between probabilistic (Bayesian) models of the value of information and heuristic search strategies, and the insights than can be gained from bridging different levels of analysis and types of models. We conclude by discussing open questions and challenges that research needs to address to build a comprehensive theory of human information acquisition.

show abstract

“…This attentional capture can be characterized in terms of information gain (Kidd, Piantadosi, & Aslin, 2012), with infants showing the most attention to situations of intermediate visual complexity, supposedly to avoid wasting cognitive resources trying to process overly simple or overly complex events (Schmidhuber, 2010). Along these lines, a growing body of work has shown that children are more likely to explore when presented with confounded (L. E. Schulz & Bonawitz, 2007) or unexpected evidence (Bonawitz, van Schijndel, Friel, & Schulz, 2012), that they seek out uncertainty reduction more eagerly than adults (Meder, Wu, Schulz, & Ruggeri, 2020;E. Schulz, Wu, Ruggeri, & Meder, 2019) and are sensitive to the potential information gain of different actions (Jirout & Klahr, 2012;Ruggeri, Sim, & Xu, 2017;Ruggeri, Swaboda, Sim, & Gopnik, 2019).…”

mentioning

confidence: 99%

Toddlers search longer when there is more information to be gained

Ruggeri¹,

Pelz²,

Gopnik³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

One of the greatest challenges for artificial intelligence is how to behave adaptively in scenarios with uncertain or no rewards. One---and perhaps the only---way to approach such complex learning problems is to build simple algorithms that grow into sophisticated adaptive agents, just like children do. But what drives children to explore and learn when external rewards are absent? Across three studies, we tested whether information gain itself acts as an internal reward and motivates children's actions. We measured 24- to 56-month-olds’ persistence in a game where they had to search for an object (animal or toy), which they never find, hidden behind a series of doors, manipulating the degree of uncertainty about \emph{which specific object} was hidden. We found that children were more persistent in their search when there was higher uncertainty, and therefore more information to be gained with each action, highlighting the importance of research on artificial intelligence to invest in curiosity-driven algorithms.

show abstract

Development of directed and random exploration in children

Cited by 63 publications

References 69 publications

Parameter and model recovery of reinforcement learning models for restless bandit problems

Parameter and model recovery of reinforcement learning models for restless bandit problems

What makes a good query? Prospects for a comprehensive theory of human information acquisition

Toddlers search longer when there is more information to be gained

Contact Info

Product

Resources

About