Advances in modeling learning and decision-making in neuroscience

Abstract

An organism's survival depends on its ability to learn about its environment and to make adaptive decisions in the service of achieving the best possible outcomes in that environment. To study the neural circuits that support these functions, researchers have increasingly relied on models that formalize the computations required to carry them out. Here, we review the recent history of computational modeling of learning and decision-making, and how these models have been used to advance understanding of prefrontal cortex function. We discuss how such models have advanced from their origins in basic algorithms of updating and action selection to increasingly account for complexities in the cognitive processes required for learning and decision-making, and the representations over which they operate. We further discuss how a deeper understanding of the real-world complexities in these computations has shed light on the fundamental constraints on optimal behavior, and on the complex interactions between corticostriatal pathways to determine such behavior. The continuing and rapid development of these models holds great promise for understanding the mechanisms by which animals adapt to their environments, and what leads to maladaptive forms of learning and decision-making within clinical populations.

Filling the gaps: Cognitive control as a critical lens for understanding mechanisms of value-based decision-making

“…For decades, research into the mechanisms of how people adjust their information processing to achieve their goals (cognitive control) and how they weigh costs and benefits to make a choice (value-based decision-making) was conducted largely in parallel, distinctions between which were seemingly underscored by differences in their dominant experimental paradigms. In a typical decision-making experiment (e.g., a choice between two gambles, food items, or consumer goods; Collins & Shenhav, 2021;Glimcher, 2002;Hare, Schultz, Camerer, O'Doherty, & Rangel, 2011;, a participant must weigh the relevant costs and benefits to determine for themselves what the best course of action is. In a typical cognitive control experiment (e.g., a Stroop, Eriksen flanker, or Simon task; Friedman & Robbins, 2021), the best course of action is indicated unambiguously (e.g., the participant is instructed to name the stimulus color, and that color is easy to identify), but choosing that response requires engaging control processes to overcome a bias towards automatically responding in a different way (e.g., based on another salient feature of the stimulus).…”

Affective prediction errors in the persistence and escalation of aggression

“…But in these cases affect is often reduced to a downstream consequence of reward prediction errors (e.g., obtaining more reward than expected makes people happier). Although prediction errors at the level of affect itself have already been documented widely in the affective forecasting literature (Nielsen et al, 2008;Wilson & Gilbert, 2005), they are less often used to predict subsequent behavior (Collins & Shenhav, 2021). A recent study rigorously addressed this gap in the context of economic games, separating emotion and reward prediction errors, finding they independently contribute to choice (Heffner et al, 2021).…”

Neural Signature of Buying Decisions in Real-World Online Shopping Scenarios – An Exploratory Electroencephalography Study Series

“…Such findings combined with knowledge about neural signaling on the molecular level further inform and put practical constraints on behavioral models of decision-making ( Johnson and Ratcliff, 2014 ). Recent decision science further shows strong advances in the development of computational models, which go far beyond original simplistic algorithms and are more and more able to account for complex decision scenarios and adaptive learning of appropriate choice behavior ( Collins and Shenhav, 2022 ).…”