Using reinforcement learning models in social neuroscience: frameworks, pitfalls, and suggestions of best practices

Zhang, Lei; Lengersdorff, Lukas; Mikuš, Nace; Gläscher, Jan; Lamm, Claus

doi:10.31234/osf.io/uthw2

Cited by 15 publications

(17 citation statements)

References 76 publications

(101 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For α close to 1, the subjective value is strongly updated by the last outcome. Note though that the learning rate α cannot directly be interpreted as a measure of how fast participants understand which of the two symbols is better (see also Zhang et al, 2020). Rather, it describes how strongly subjective values are influenced by the last outcome, regardless of the outcomes that came before.…”

Section: Models For Outcome Evaluationmentioning

confidence: 99%

When Implicit Prosociality Trumps Selfishness: The Neural Valuation System Underpins More Optimal Choices When Learning to Avoid Harm to Others Than to Oneself

et al. 2020

Self Cite

View full text Add to dashboard Cite

This work was funded by the Vienna Science and Technology Fund (WWTF VRG13-007). We would like to thank Patricia Lockwood for feedback regarding the experimental task; Lei Zhang and Nace Mikus for helpful discussions concerning the modeling analyses, and feedback on the manuscript; and David Sastre-Yagüe, Gloria Mittmann, André Lüttig, Sophia Shea and Leonie Brög for assistance during data collection. Significance Statement We quickly learn to avoid actions that cause us harm. As "social animals", we also need to learn and consider the harmful consequences our actions might have for others. Here, we investigated how learning to protect others from pain (prosocial learning) differs from learning to protect oneself (selfrelevant learning). We found that human participants performed better during prosocial learning than during self-relevant learning, as they were more sensitive towards the information they collected when making choices for the other. Prosocial learning recruited similar brain areas as self-relevant learning, but additionally involved parts of the "social brain" that underpin perspective-taking and self-other distinction. Our findings support the notion that humans have deeply engrained mechanisms tailoring them towards prosociality.

show abstract

Section: Models For Outcome Evaluationmentioning

confidence: 99%

When Implicit Prosociality Trumps Selfishness: The Neural Valuation System Underpins More Optimal Choices When Learning to Avoid Harm to Others Than to Oneself

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, they may also wish to consult many other reviews and excellent guidelines on the topic ( Daw and Doya, 2006 ; Dayan and Niv, 2008 ; Samson et al. , 2010 ; Daw, 2011 ; Zhang et al. , 2019 ).…”

Section: Introductionmentioning

confidence: 99%

Computational modelling of social cognition and behaviour—a reinforcement learning primer

Lockwood

Klein-Flügge

2020

Social Cognitive and Affective Neuroscience

View full text Add to dashboard Cite

Abstract Social neuroscience aims to describe the neural systems that underpin social cognition and behaviour. Over the past decade, researchers have begun to combine computational models with neuroimaging to link social computations to the brain. Inspired by approaches from reinforcement learning theory, which describes how decisions are driven by the unexpectedness of outcomes, accounts of the neural basis of prosocial learning, observational learning, mentalizing and impression formation have been developed. Here we provide an introduction for researchers who wish to use these models in their studies. We consider both theoretical and practical issues related to their implementation, with a focus on specific examples from the field.

show abstract

“…A changeable environment requires fast learning guided by recent feedback, whereas a stable environment requires slower learning over time (e.g. [35,36]). Crucially, probabilistic feedback also requires learning to ignore 'misleading' punishment.…”

Section: Introductionmentioning

confidence: 99%

Modeling flexible behaviour in autism spectrum disorder and typical development

Crawley¹,

Zhang²,

Jones³

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

Flexible behavior is critical for everyday decision-making and has been implicated in restricted, repetitive behaviors (RRB) in autism spectrum disorder (ASD). However, how flexible behavior changes developmentally in ASD remains largely unknown. Here, we used a developmental approach and examined flexible behavior on a probabilistic reversal learning task in 572 children, adolescents and adults (ASD N=321; typical development, TD; N=251). Using computational modeling, we quantified latent variables that index mechanisms underlying perseveration and feedback sensitivity. We then assessed these variables in relation to diagnosis, developmental stage, core autism symptomatology and associated psychiatric symptoms. Autistic individuals showed on average more perseveration and less feedback sensitivity than TD individuals and, across cases and controls, older age groups showed more feedback sensitivity than younger age groups. Computational modeling revealed that dominant learning mechanisms underpinning flexible behavior differed across developmental stages and reduced flexible behavior in ASD was driven by less optimal learning on average within each age group. In autistic children, perseverative errors were positively related to anxiety symptoms, and in autistic adults, perseveration (indexed by both task errors and model parameters) was positively related to RRB. These findings provide novel insights into reduced flexible behavior in relation to clinical symptoms in ASD.

show abstract

Using reinforcement learning models in social neuroscience: frameworks, pitfalls, and suggestions of best practices

Cited by 15 publications

References 76 publications

When Implicit Prosociality Trumps Selfishness: The Neural Valuation System Underpins More Optimal Choices When Learning to Avoid Harm to Others Than to Oneself

When Implicit Prosociality Trumps Selfishness: The Neural Valuation System Underpins More Optimal Choices When Learning to Avoid Harm to Others Than to Oneself

Computational modelling of social cognition and behaviour—a reinforcement learning primer

Modeling flexible behaviour in autism spectrum disorder and typical development

Contact Info

Product

Resources

About