Probing relationships between reinforcement learning and simple behavioral strategies to understand probabilistic reward learning

Iyer, Eshaan; Kairiss, Megan A.; Liu, Adrian; Otto, A. Ross; Bagot, Rosemary C.

doi:10.1016/j.jneumeth.2020.108777

Cited by 4 publications

(3 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To learn how the skiers and coaches used feedback to guide their choices, we constructed a 'win-stay, lose-shift' model [56][57][58]. For this analysis, we z-scored the race times for each skier for Free Choices 1 and 2 and counted y ijk = 1 when the skier repeated the previous strategy and 0 when they did not.…”

Section: Strategy Choicesmentioning

confidence: 99%

“…Our hypothesis was that, compared with coaches, skiers in the reinforcement learning group would engage more actively with these race times to improve their choices. To test this hypothesis, we conducted a 'win-stay, lose-shift' analysis, where the primary assumption is that all information used for decision-making stems from the last trial (n-1) [63,64]. In this analysis, heightened sensitivity to the most recent feedback is reflected by a high predicted probability of repeating an action following positive feedback (a fast time, compared to the average time) and a low predicted probability following negative feedback (a slow time, compared to the average time) on the preceding trial.…”

Section: Similar Use Of Feedback For Strategy Selection In Reinforcem...mentioning

confidence: 99%

See 1 more Smart Citation

Reinforcement learning enhances training and performance in skilled alpine skiers compared to traditional coaching instruction

Magelssen,

Gilgien,

Tajet

et al. 2024

Preprint

View full text Add to dashboard Cite

Skilled performers need skillful and adaptive movement strategies to solve tasks effectively. Typically, performers learn these strategies with instruction-based teaching methods where coaches offer performers a correct solution. Inspired by recent evidence from decision neuroscience, we asked whether skilled performers learn strategy choices better with an evaluation-based training strategy (reinforcement learning). To address this question, we conducted a three-day learning experiment with skilled alpine ski racers (n=98) designed to improve their performance on flat slopes on slaloms with four strategies at their disposal to achieve this goal. We compared performance and strategy choices of three groups: a reinforcement learning group, that only received feedback about their race times after every run, a supervised (free choice) learning group, that received strategy instructions from their coach, and a supervised (target skill) learning group, being coached to use the theoretically optimal strategy for skiing well on flats. We found that despite making similar strategy choices, the skiers in the reinforcement learning group, showed greater improvements in their race times during the training sessions than their counterparts in the supervised (free choice) learning group and outperformed them during a subsequent retention test. Surprisingly, the skiers in the reinforcement learning group even showed descriptively (but not significantly) better performance than those in the supervised (target skill) learning group. Our findings show that reinforcement learning can be an effective training strategy for improving strategy choices and performance among skilled performers, even among the best ones.

show abstract

Section: Strategy Choicesmentioning

confidence: 99%

Section: Similar Use Of Feedback For Strategy Selection In Reinforcem...mentioning

confidence: 99%

Reinforcement learning enhances training and performance in skilled alpine skiers compared to traditional coaching instruction

Magelssen,

Gilgien,

Tajet

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Psychologists and economists have developed a range of theoretical models for modeling decision-making processes 6 , 10 , 11 . Win-Stay-Lose-Shift (WSLS) models have been extensively used to model behavior in decision-making tasks, especially from binary choice experiments 12 , 13 .…”

mentioning

confidence: 99%

Switching costs in stochastic environments drive the emergence of matching behaviour in animal decision-making through the promotion of reward learning strategies

Lyu

Zhang

et al. 2021

Sci Rep

View full text Add to dashboard Cite

A principle of choice in animal decision-making named probability matching (PM) has long been detected in animals, and can arise from different decision-making strategies. Little is known about how environmental stochasticity may influence the switching time of these different decision-making strategies. Here we address this problem using a combination of behavioral and theoretical approaches, and show, that although a simple Win-Stay-Loss-Shift (WSLS) strategy can generate PM in binary-choice tasks theoretically, budgerigars (Melopsittacus undulates) actually apply a range of sub-tactics more often when they are expected to make more accurate decisions. Surprisingly, budgerigars did not get more rewards than would be predicted when adopting a WSLS strategy, and their decisions also exhibited PM. Instead, budgerigars followed a learning strategy based on reward history, which potentially benefits individuals indirectly from paying lower switching costs. Furthermore, our data suggest that more stochastic environments may promote reward learning through significantly less switching. We suggest that switching costs driven by the stochasticity of an environmental niche can potentially represent an important selection pressure associated with decision-making that may play a key role in driving the evolution of complex cognition in animals.

show abstract

Impairments in operant probabilistic reversal learning in BTBR T+tf/J male and female mice

Alvarez

Morales²,

Oliver³

et al. 2023

Behavioural Brain Research

View full text Add to dashboard Cite

Probing relationships between reinforcement learning and simple behavioral strategies to understand probabilistic reward learning

Cited by 4 publications

References 27 publications

Reinforcement learning enhances training and performance in skilled alpine skiers compared to traditional coaching instruction

Reinforcement learning enhances training and performance in skilled alpine skiers compared to traditional coaching instruction

Switching costs in stochastic environments drive the emergence of matching behaviour in animal decision-making through the promotion of reward learning strategies

Impairments in operant probabilistic reversal learning in BTBR T+tf/J male and female mice

Contact Info

Product

Resources

About