Dopamine Modulates Dynamic Decision-Making during Foraging

Abstract

The mesolimbic dopaminergic system exerts a crucial influence on incentive processing. However, the contribution of dopamine in dynamic, ecological situations where reward rates vary, and decisions evolve over time, remains unclear. In such circumstances, current (foreground) reward accrual needs to be compared continuously with potential rewards that could be obtained by traveling elsewhere (background reward rate), to determine the opportunity cost of staying versus leaving. We hypothesized that dopamine specifically modulates the influence of background, but not foreground, reward information when making a dynamic comparison of these variables for optimal behavior. On a novel foraging task based on an ecological account of animal behavior (marginal value theorem), human participants of either sex decided when to leave locations in situations where foreground rewards depleted at different rates, either in rich or poor environments with high or low background reward rates. In line with theoretical accounts, people's decisions to move from current locations were independently modulated by changes in both foreground and background reward rates. Pharmacological manipulation of dopamine D2 receptor activity using the agonist cabergoline significantly affected decisions to move on, specifically modulating the effect of background reward rates. In particular, when on cabergoline, people left patches in poor environments much earlier. These results demonstrate a role of dopamine in signaling the opportunity cost of rewards, not value per se. Using this ecologically derived framework, we uncover a specific mechanism by which D2 dopamine receptor activity modulates decision-making when foreground and background reward rates are dynamically compared.

Control over patch encounters changes foraging behaviour

“…Third, the experiment could be limited in its trial-based structure. While some patch-leaving designs are trial-based (Barack et al, 2017; Constantino & Daw, 2015; Kane et al, 2017; Hayden et al, 2011), studies tend to compute reward rates (Kane et al, 2017; Lottem et al, 2018), the probability of leaving (Constantino & Daw, 2015) or leaving times (Barack et al, 2017; Hayden et al, 2011; Le Heron et al, 2020; Lottem et al, 2018) using some continuous time information, such as the number of seconds in a patch. In the present experiment, we used trial-wise data to model discrete choice behaviour based on a previous computational model (Constantino & Daw, 2015).…”

“…This results in a simple optimal policy called the Marginal Value Theorem (MVT, Charnov, 1976), wherein animals should leave their current option when its reward falls below the average reward rate for the environment. Empirical studies have shown that leave decisions in mice, rats, monkeys and humans are consistent with this decision rule (Constantino & Daw, 2015, Hayden et al, 2011; Kane et al, 2017; Le Heron et al, 2020; Lottem et al 2018). Evidence has also suggested that the unique structure of foraging decisions might be solved using neural substrates that are at least partially distinct from those involved in other reward-based decisions (Kolling et al, 2012; Rushworth et al, 2011), with the anterior cingulate cortex serving a critical role in regulating decisions about whether to disengage from an option (Fouragnan et al, 2019; Hayden et al, 2011; Wittmann et al, 2016, see Kolling et al, 2016; Mobbs et al, 2018; Rushworth et al, 2011 for reviews).…”

“…Human participants completed a patch-leaving task similar to studies from Optimal Foraging Theory (Stephens & Krebs, 1986), in which an option’s reward decreased over time through exploitation. In contrast to previous studies (Constantino & Daw, 2015, Hayden et al, 2011; Kane et al, 2017; Le Heron et al, 2020), participants switched between the same three options throughout a block, which replenished their rewards at distinct rates when not being exploited. Participants therefore needed to decide when to leave their current option in order to revisit and exploit one of the alternatives.…”

Decision Neuroscience and Neuroeconomics: Recent Progress and Ongoing Challenges

“…Newer evidence has suggested that subcortical areas and especially those implicated in dopaminergic transmission have a more substantial role in explore-exploit behavior than previously realized (Chakroun et al, 2020). Increased dopamine in the striatum promoted explorative behavior (Verharen et al, 2019), influenced participants to leave patches in poor environments earlier in a foraging task (Heron et al, 2020), attenuated directed exploration, and lowered representations of overall uncertainty in the insula and dACC (Chakroun et al, 2020). However, a major factor in explore-exploit decisions is the environmental uncertainty of new options (Badre et al, 2012;Navarro et al, 2016;Tomov et al, 2020), information which seems to be largely processed in the PFC.…”

Neuromodulation of Foraging Decisions: The Role of Dopamine

“…In their recent study, Le Heron et al (2020) fill this gap by investigating how these decisions are causally affected by dopaminergic state in an ecologically valid foraging scenario. In their study, participants could choose between collecting reward (milk filling a bucket) at one location (patch) or leaving for another patch which incurred a cost in the form of a fixed travel time.…”