Beliefs about Others’ Abilities Alter Learning from Observation

Selbing, Ida; Olsson, Andreas

doi:10.1038/s41598-017-16307-3

Cited by 16 publications

(15 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, we compared the imitation learning rates across different observational condition (Skilled vs. Unskilled Demonstrators), to see whether imitation is modulated. Consistent with our model comparison results and with previous theoretical and empirical work about meta-learning (8,14), we found that subjects can infer the skill of a Demonstrator and regulate imitation accordingly. This result was even more pronounced when the skill of the Demonstrator was implemented within-subject, which suggests that being exposed to both Skilled and Unskilled Demonstrators models could be more effective in preventing Learners from bad influence.…”

Section: Introductionsupporting

confidence: 91%

“…Here we focused on the Demonstrator's skills and found that when the Demonstrator was not skilled, the imitation rate was downregulated. In other terms, consistent with previous studies, we show that the adaptive regulation of imitation can be achieved by monitoring endogenous signals (likely an estimate of the Demonstrator's performance) and does not necessary require explicit cues and instructions (7,8,26). At the computational level, this effect can be formalized as a metalearning process, where relevant variables are used to optimally tune the learning parameters (27,28).…”

Section: Value Shaping Vs Decision Biasing Decision Biasingsupporting

confidence: 85%

“…While these studies provide valuable insights about the computational mechanisms of imitation, they are still limited in their scope as imitation is treated in isolation from other learning processes, such as autonomous -reinforcement -learning. Here we compare three radically different and psychologically plausible computational implementations of how imitation can be integrated into a standard reinforcement learning algorithm (6)(7)(8)(9). To illustrate the three hypotheses we consider the stylized situation, where a reinforcement Leaner is exposed to the choices of a Demonstrator, before making her own choices.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Imitation as a model-free process in human reinforcement learning

Najar

Bonnet

Bahrami

et al. 2019

Preprint

View full text Add to dashboard Cite

While there is not doubt that social signals affect human reinforcement learning, there is still no consensus about their exact computational implementation. To address this issue, we compared three hypotheses about the algorithmic implementation of imitation in human reinforcement learning. A first hypothesis, decision biasing, postulates that imitation consists in transiently biasing the learner's action selection without affecting her value function. According to the second hypothesis, model-based imitation, the learner infers the demonstrator's value function through inverse reinforcement learning and uses it for action selection. Finally, according to the third hypothesis, value shaping, demonstrator's actions directly affect the learner's value function. We tested these three psychologically plausible hypotheses in two separate experiments (N = 24 and N = 44) featuring a new variant of a social reinforcement learning task, where we manipulated the quantity and the quality of the demonstrator's choices. We show through model comparison that value shaping is favored, which provides a new perspective on how imitation is integrated into human reinforcement learning. Reinforcement Learning | Social Learning | Imitation | Computational cognitive modeling | Decision-making | Meta-learningCorrespondence: anis.najar@ens.fr, stefano.palminteri@ens.fr

show abstract

Section: Introductionsupporting

confidence: 91%

Section: Value Shaping Vs Decision Biasing Decision Biasingsupporting

confidence: 85%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Imitation as a model-free process in human reinforcement learning

Najar

Bonnet

Bahrami

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…While these studies provide valuable insights about the computational mechanisms of imitation, they are still limited in their scope as imitation is treated in isolation from other learning processes, such as autonomous reinforcement learning. Here, we compare three radically different and psychologically plausible computational implementations of how imitation can be integrated into a standard reinforcement learning algorithm [6][7][8][9]. To illustrate the three hypotheses, we consider the stylized situation, where a reinforcement Learner is exposed to the choices of a Demonstrator, before making her own choices.…”

Section: Introductionmentioning

confidence: 99%

The actions of others act as a pseudo-reward to drive imitation in the context of social reinforcement learning

et al. 2020

View full text Add to dashboard Cite

While there is no doubt that social signals affect human reinforcement learning, there is still no consensus about how this process is computationally implemented. To address this issue, we compared three psychologically plausible hypotheses about the algorithmic implementation of imitation in reinforcement learning. The first hypothesis, decision biasing (DB), postulates that imitation consists in transiently biasing the learner’s action selection without affecting their value function. According to the second hypothesis, model-based imitation (MB), the learner infers the demonstrator’s value function through inverse reinforcement learning and uses it to bias action selection. Finally, according to the third hypothesis, value shaping (VS), the demonstrator’s actions directly affect the learner’s value function. We tested these three hypotheses in 2 experiments (N = 24 and N = 44) featuring a new variant of a social reinforcement learning task. We show through model comparison and model simulation that VS provides the best explanation of learner’s behavior. Results replicated in a third independent experiment featuring a larger cohort and a different design (N = 302). In our experiments, we also manipulated the quality of the demonstrators’ choices and found that learners were able to adapt their imitation rate, so that only skilled demonstrators were imitated. We proposed and tested an efficient meta-learning process to account for this effect, where imitation is regulated by the agreement between the learner and the demonstrator. In sum, our findings provide new insights and perspectives on the computational mechanisms underlying adaptive imitation in human reinforcement learning.

show abstract

“…However, participants receive only probabilistic information about choice-outcome associations, so choosing optimally requires dynamic learning about the environment. During observational learning, instead of choosing actions and receiving outcomes themselves, participants observe other individuals selecting between available options for rewards (Hill, Boorman, & Fried, 2016;Selbing & Olsson, 2017). From observed choice-outcome associations, participants vicariously learn the underlying reward distributions.…”

Section: Observational Learning Tasksmentioning

confidence: 99%

Theory of mind and decision science: Towards a typology of tasks and computational models

et al. 2020

View full text Add to dashboard Cite

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

show abstract

Beliefs about Others’ Abilities Alter Learning from Observation

Cited by 16 publications

References 54 publications

Imitation as a model-free process in human reinforcement learning

Imitation as a model-free process in human reinforcement learning

The actions of others act as a pseudo-reward to drive imitation in the context of social reinforcement learning

Theory of mind and decision science: Towards a typology of tasks and computational models

Contact Info

Product

Resources

About