Impulsivity Relates to Multi-Trial Choice Strategy in Probabilistic Reversal Learning

Abstract

Impulsivity is defined as a trait-like tendency to engage in rash actions that are poorly thought out or expressed in an untimely manner. Previous research has found that impulsivity relates to deficits in decision making, in particular when it necessitates executive control or reward outcomes. Reinforcement learning (RL) relies on the ability to integrate reward or punishment outcomes to make good decisions, and has recently been shown to often recruit executive function; as such, it is unsurprising that impulsivity has been studied in the context of RL. However, how impulsivity relates to the mechanisms of RL remains unclear. We aimed to investigate the relationship between impulsivity and learning in a reward-driven learning task with probabilistic feedback and reversal known to recruit executive function. Based on prior literature in clinical populations, we predicted that higher impulsivity would be associated with poorer performance on the task, driven by more frequent switching following unrewarded outcomes. Our results did not support this prediction, but more advanced, trial-history dependent analyses revealed specific effects of impulsivity on switching behavior following consecutive unrewarded trials. Computational modeling captured group-level behavior, but not impulsivity results. Our results support previous findings highlighting the importance of sensitivity to negative outcomes in understanding how impulsivity relates to learning, but indicate that this may stem from more complex strategies than usually considered in computational models of learning. This should be an important target for future research.

Artificial neural networks for model identification and parameter estimation in computational cognitive models

“…• 3-back feedback integration: The 3-back analysis quantifies learning as well; however, instead of aligning the performance to reversal points, it allowed us to examine agents' tendency to repeat action selection from the previous trial contingent on reward history-specifically the outcome they observed on the most recent 3 trials. Higher probability of repeating the same action following more positive feedback indicates learning/sensitivity to reward as reported in [11] • Ab-analysis: The Ab-analysis allowed us to quantify probability of selecting an action at trial t,contingent both on previous reward and action selection history (trials t − 2 and t − 1, [11,80]).…”

“…Furthermore, by estimating model parameters that quantify underlying cognitive processes, researchers have been able to characterize important individual differences (e.g. developmental: [7][8][9][10]; clinical: [11][12][13][14][15]) as well as condition effects [16,17].…”

Artificial neural networks for model identification and parameter estimation in computational cognitive models

“…This summary statistic should capture learning dynamics, as the agent learns to select the correct action, and then experiences dip in accuracy once the correct actions switch, subsequently learning to adjust based on feedback after several trials. 3-back feedback integration: The 3-back analysis quantifies learning as well; however, instead of aligning the performance to reversal points, it allowed us to examine agents’ tendency to repeat action selection from the previous trial contingent on reward history - specifically the outcome they observed on the most recent 3 trials. Higher probability of repeating the same action following more positive feedback indicates learning/sensitivity to reward as reported in Zou et al, 2022 Ab-analysis: The Ab-analysis allowed us to quantify probability of selecting an action at trial t ,contingent both on previous reward and action selection history (trials t − 2 and t − 1, Beron et al, 2021; Zou et al, 2022). For the models simulated on a hierarchical task we used the learning curves as summary statistics (same as for the PRL), where reversal points were defined as the switch of the correct rule/arrow to follow.…”

“…Higher probability of repeating the same action following more positive feedback indicates learning/sensitivity to reward as reported in Zou et al, 2022 • Ab-analysis: The Ab-analysis allowed us to quantify probability of selecting an action at trial t,contingent both on previous reward and action selection history (trials t − 2 and t − 1, Beron et al, 2021;Zou et al, 2022).…”

“…developmental: Eppinger et al, 2013; Hauser et al, 2015; Nussenbaum et al, 2022; Rmus et al, 2023; clinical: C. Chen et al, 2015; Collins et al, 2014; Gillan et al, 2016; Peterson et al, 2009; Zou et al, 2022) as well as condition effects (Sheynin et al, 2015; Weber et al, 2022).…”