Learning from other minds: an optimistic critique of reinforcement learning models of social learning

Vélez, Natalia; Gweon, Hyowon

doi:10.1016/j.cobeha.2021.01.006

Cited by 43 publications

(35 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Firstly, the paired participants consistently outperformed the solo participants in all six spatially-correlated 165-armed bandit environments (Fig. 2a & 2b), replicating the performance improvement due to social learning in simpler MAB environments without spatial correlations 34,35 . More central to our argument, the superiority of the paired participants to the solo participants was more pronounced in later sessions (Fig.…”

Section: Discussionmentioning

confidence: 56%

“…This result indicates that the social interaction between decision makers played an essential role in developing insights about the generative rule. In other words, across-task learning may be achieved as a kind of collective intelligence, emerging through reciprocal interactions of learning with a partner, rather than one-way observational learning [34][35][36] .…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Insights about the common generative rule underlying an information foraging task can be facilitated via collective search

Naito¹,

Katahira²,

Kameda³

2022

Preprint

View full text Add to dashboard Cite

Social learning is beneficial for efficient information search in unfamiliar environments (“within-task” learning). In the real world, however, possible search spaces are often so large that decision makers are incapable of covering all options, even if they pool their information collectively. One strategy to handle such overload is developing generalizable knowledge that extends to multiple related environments (“across-task” learning). However, it is unknown whether and how social information may facilitate such across-task learning. Here, we investigated participants’ social learning processes across multiple laboratory foraging sessions in spatially correlated reward landscapes that were generated according to a common rule. The results showed that paired participants were able to improve efficiency in information search across sessions more than solo participants. Computational analysis of participants’ choice-behaviors revealed that such improvement across sessions was related to better understanding of the common generative rule. Rule understanding was correlated within a pair, suggesting that social interaction is a key to the improvement of across-task learning.

show abstract

Section: Discussionmentioning

confidence: 56%

Section: Discussionmentioning

confidence: 99%

Insights about the common generative rule underlying an information foraging task can be facilitated via collective search

Naito¹,

Katahira²,

Kameda³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…While IRL illuminates how humans reason about others' preferences, goals, and beliefs (Collette et al, 2017;Baker, Jara-Ettinger, Saxe, & Tenenbaum, 2017;Jern et al, 2017), it is computationally costly. For most interesting problems IRL is computationally intractable (Jara-Ettinger, Gweon, Schulz, & Tenenbaum, 2016;Vélez & Gweon, 2021). Nevertheless, as a rational framework it can be used to uncover inductive biases that simplify the required computations.…”

Section: Value Inference and Cached Valuementioning

confidence: 99%

“…To some, the idea that humans arbitrate among social learning mechanisms may seem puzzling. Humans can make incredible inductive leaps from sparse social information by reasoning about the beliefs and values of other people (Vélez & Gweon, 2021). Why not always use the most sophisticated instrument in our repertoire?…”

Section: Arbitrationmentioning

confidence: 99%

Representational exchange in human social learning: Balancing efficiency and flexibility

Cm¹,

Vélez²,

Cushman³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

What makes human social learning so powerful? While past accounts have sometimes prioritized finding the single capacity that makes the largest difference, our social learning abilities span a wide spectrum of capacities from the high-fidelity imitation of behaviors to inferring and learning from hidden mental states. Here, we propose that the power of human social learning lies not within a single capacity, but in our ability to flexibly arbitrate between different computations and to integrate their outputs. In particular, learners can directly copy the demonstrator’s actions in the absence of causal insight (policy imitation), infer their instrumental values (value inference), or infer their model of the world and intrinsic rewards (belief inference and reward inference). Each of these strategies trades off the cost of computation against the flexibility and compositionality of its outputs. Crucially, we have the capacity to arbitrate and exchange information between these representational formats. Human social learning, we suggest, is powerful not just because of the way it moves information between minds, but also because of the way it flexibly moves information within them.

show abstract

“…Learning from human interactions will therefore allow agents to learn what we think is important about the world, and how to communicate with us about it, and ultimately how to construct new knowledge beyond what is known. There is much to be explored at the interface of AI and social learning (Vélez and Gweon, 2020).…”

Section: How Can Symbolic Behaviour Come About?mentioning

confidence: 99%

Symbolic Behaviour in Artificial Intelligence

Santoro¹,

Lampinen²,

Mathewson³

et al. 2021

Preprint

View full text Add to dashboard Cite

The ability to use symbols is the pinnacle of human intelligence, but has yet to be fully replicated in machines. Here we argue that the path towards symbolically fluent artificial intelligence (AI) begins with a reinterpretation of what symbols are, how they come to exist, and how a system behaves when it uses them. We begin by offering an interpretation of symbols as entities whose meaning is established by convention. But crucially, something is a symbol only for those who demonstrably and actively participate in this convention. We then outline how this interpretation thematically unifies the behavioural traits humans exhibit when they use symbols. This motivates our proposal that the field place a greater emphasis on symbolic behaviour rather than particular computational mechanisms inspired by more restrictive interpretations of symbols. Finally, we suggest that AI research explore social and cultural engagement as a tool to develop the cognitive machinery necessary for symbolic behaviour to emerge. This approach will allow for AI to interpret something as symbolic on its own rather than simply manipulate things that are only symbols to human onlookers, and thus will ultimately lead to AI with more human-like symbolic fluency.

show abstract

Learning from other minds: an optimistic critique of reinforcement learning models of social learning

Cited by 43 publications

References 71 publications

Insights about the common generative rule underlying an information foraging task can be facilitated via collective search

Insights about the common generative rule underlying an information foraging task can be facilitated via collective search

Representational exchange in human social learning: Balancing efficiency and flexibility

Symbolic Behaviour in Artificial Intelligence

Contact Info

Product

Resources

About