Expected Value of Control and the Motivational Control of Habitual Action

Abstract

A hallmark of habitual actions is that, once they are established, they become insensitive to changes in the values of action outcomes. In this article, we review empirical research that examined effects of posttraining changes in outcome values in outcome-selective Pavlovian-to-instrumental transfer (PIT) tasks. This review suggests that cue-instigated action tendencies in these tasks are not affected by weak and/or incomplete revaluation procedures (e.g., selective satiety) and substantially disrupted by a strong and complete devaluation of reinforcers. In a second part, we discuss two alternative models of a motivational control of habitual action: a default-interventionist framework and expected value of control theory. It is argued that the default-interventionist framework cannot solve the problem of an infinite regress (i.e., what controls the controller?). In contrast, expected value of control can explain control of habitual actions with local computations and feedback loops without (implicit) references to control homunculi. It is argued that insensitivity to changes in action outcomes is not an intrinsic design feature of habits but, rather, a function of the cognitive system that controls habitual action tendencies.

The role of the bed nucleus of the stria terminalis in the motivational control of instrumental action

“…Of these sources of predicted value, there have been several recent reviews of those controlling the influence of identity-specific reward predictions on instrumental performance, focussing mostly on their function in action selection in outcome-specific Pavlovian-instrumental transfer (Holmes et al, 2010 ; Cartoni et al, 2016 ; Watson et al, 2018 ; Balleine, 2019 ; Eder and Dignath, 2019 ; Laurent and Balleine, 2021 ). The current article is instead concerned with the contribution of general incentive processes to performance, i.e., those that induce their effects through a form of energetic shift in motivational or affective arousal.…”

Macaques fail to develop habit responses during extended training on a reinforcer devaluation task.

“…Habits, on the other hand, are typically characterized by inflexibility and insensitivity to updated value information. The functional distinction between habitual and goal-directed action is more gradual than categorical (Eder & Dignath, 2019), as repetition of goal-directed actions may transform them into habit; this transition is incremental, unconscious, and automatic (Seger & Spiering, 2011). In the healthy brain, the goal-directed system and the habit-based system are parallel, complementary, bidirectional processes (Lipton et al, 2019) critical for mental and emotional well-being (Verplanken & Sui, 2019).…”