The Balloon Analogue Risk Task (BART) is a popular task used to measure risk-taking behavior. To identify cognitive processes associated with choice behavior on the BART, a few computational models have been proposed. However, the extant models either fail to capture choice patterns on the BART or show poor parameter recovery performance. Here, we propose a novel computational model, the exponential-weight mean-variance (EWMV) model, which addresses the limitations of existing models.By using multiple model comparison methods, including post hoc model fits criterion and parameter recovery, we showed that the EWMV model outperforms the existing models. In addition, we applied the EWMV model to BART data from healthy controls and substance-using populations (patients with past opiate and stimulant dependence). The results suggest that (1) the EWMV model addresses the limitations of existing models and (2) heroin-dependent individuals show reduced risk preference than other groups, which may have significant clinical implications.

The Balloon Analogue Risk Task (BART) is a popular task used to measure risk-taking behavior. To identify cognitive processes associated with choice behavior on the BART, a few computational models have been proposed. However, the existing models are either too simplistic or fail to show good parameter recovery performance. Here, we propose a novel computational model, the exponential weight updating (EU) model, which addresses the limitations of existing models. By using multiple model comparison methods including post hoc model fits criterion and parameter recovery, we showed that the EU model outperforms the existing models. In addition, we applied the EU model to BART data from healthy controls and substance-using populations (patients with past opiate and stimulant dependence). The results suggest that (1) the EU model addresses the limitations of existing models and (2) heroin-dependent individuals show increased risk preference and reduced loss aversion than other groups, which may have significant clinical implications.

No abstract

Humans and animals learn optimal behaviors by interacting with the environment. Research suggests that a fast, capacity-limited working memory (WM) system and a slow, incremental reinforcement learning (RL) system jointly contribute to instrumental learning. Situations that strain WM resources alter several decision-making processes and the balance between multiple decision-making systems: under WM loads, learning becomes slow and incremental, while reward prediction error (RPE) signals become stronger; the reliance on computationally efficient learning increases as WM demands are balanced against computationally costly strategies; and action selection becomes more random. Meanwhile, instrumental learning is known to interact with Pavlovian learning, a hard-wired system that motivates approach to reward and avoidance of punishment. However, the neurocognitive role of WM load during instrumental learning under Pavlovian influence remains unknown, while conflict between the two systems sometimes leads to suboptimal behavior. Thus, we conducted a functional magnetic resonance imaging (fMRI) study (N = 49) in which participants completed an instrumental learning task with Pavlovian-instrumental conflict (the orthogonalized go/no-go task); WM load was manipulated with dual-task conditions. Behavioral and computational modeling analyses revealed that WM load compromised learning by reducing the learning rate and increasing random choice, without affecting Pavlovian bias. Model-based fMRI analysis revealed that WM load strengthened RPE signaling in the striatum. Moreover, under WM load, the striatum showed weakened connectivity with the ventromedial and dorsolateral prefrontal cortex when computing reward expectations. These results suggest that the limitation of cognitive resources by WM load decelerates instrumental learning through the weakened cooperation between WM and RL; such limitation also makes action selection more random, but it does not directly affect the balance between instrumental and Pavlovian systems.