Action Dominates Valence in Anticipatory Representations in the Human Striatum and Dopaminergic Midbrain

Abstract

The acquisition of reward and the avoidance of punishment could logically be contingent on either emitting or withholding particular actions. However, the separate pathways in the striatum for go and no-go appear to violate this independence, instead coupling affect and effect. Respect for this interdependence has biased many studies of reward and punishment, so potential action-outcome valence interactions during anticipatory phases remain unexplored. In a functional magnetic resonance imaging study with healthy human volunteers, we manipulated subjects' requirement to emit or withhold an action independent from subsequent receipt of reward or avoidance of punishment. During anticipation, in the striatum and a lateral region within the substantia nigra/ventral tegmental area (SN/VTA), action representations dominated over valence representations. Moreover, we did not observe any representation associated with different state values through accumulation of outcomes, challenging a conventional and dominant association between these areas and state value representations. In contrast, a more medial sector of the SN/VTA responded preferentially to valence, with opposite signs depending on whether action was anticipated to be emitted or withheld. This dominant influence of action requires an enriched notion of opponency between reward and punishment.

Neural basis of decision making guided by emotional outcomes

“…In addition, our results are consistent with Levita et al (2012), who reported that the BOLD responses from the ventral striatum are higher in active avoidance conditions compared with passive avoidance conditions. A series of fMRI studies reported that the BOLD responses in striatum are dominated by action requirements (e.g., go vs. no go) rather than by valence, in accord with the view that dopamine has a role in modulating vigor or motivation for actions independent of valence (Guitart-Masip et al 2011. We considered two types of action-value representations.…”

Action controls dopaminergic enhancement of reward representations

“…Such conflicting cases have been a focal point of recent studies where the critical manipulation involved an orthogonalization of action requirements and outcome valence (15,18). In the latter study we highlighted anticipatory representations in the striatum and substantia nigra/ventral tegmental area (SN/VTA) that reflected a dominance of action representation over an expected dominance of reward value representations, thereby hinting at a specific role for dopamine in motivation for action (18).…”

“…In the latter study we highlighted anticipatory representations in the striatum and substantia nigra/ventral tegmental area (SN/VTA) that reflected a dominance of action representation over an expected dominance of reward value representations, thereby hinting at a specific role for dopamine in motivation for action (18).…”

“…That is, although the nature of any opponency between appetitive and aversive systems remains the subject of much debate (12-14), there are suggestions that serotonin acts as a mirror to dopamine and is associated with behavioral inhibition in aversive contexts (13-17). These joint roles would lead to conflict when punishment can be avoided only by acting (go).Such conflicting cases have been a focal point of recent studies where the critical manipulation involved an orthogonalization of action requirements and outcome valence (15,18). In the latter study we highlighted anticipatory representations in the striatum and substantia nigra/ventral tegmental area (SN/VTA) that reflected a dominance of action representation over an expected dominance of reward value representations, thereby hinting at a specific role for dopamine in motivation for action (18).…”

Fatal attraction: Ventral striatum predicts costly choice errors in humans

“…Simple Pavlovian learning does not provide a full account of even animal choice learning and is often viewed as just one factor influencing behaviour (Seward 1949;Thorpe 1956;Dayan and Balleine 2002;Dayan, Niv et al 2006;Talmi, Seymour et al 2008;Guitart-Masip, Fuentemilla et al 2011;Prévost, Liljeholm et al 2012). More sophisticated learning capabilities, each with their own quirks, underpin our habitual and goal-directed actions: these systems may not even require "prediction error" mechanisms.…”