Deeply Felt Affect: The Emergence of Valence in Deep Active Inference

Abstract

The positive-negative axis of emotional valence has long been recognised as fundamental to adaptive behaviour, but its domain-generality has largely eluded formal theories and modelling. Using deep active inference – a hierarchical inference scheme that rests on inverting a model of how sensory data are generated – we develop a principled Bayesian account of emotional valence. This formulation associates valence with subjective fitness and exploits the domain-generality of second-order beliefs (i.e., beliefs about beliefs). We construct an affective agent that infers its valence state from the expected precision of its phenotype-congruent action model (i.e., subjective fitness) in any given environment. The ensuing affective states then optimise that confidence pre-emptively. The evidence for inferred (i.e., ‘felt’) valenced states depends upon the (Bayes-optimal) updating term for subjective fitness, which we label affective charge (AC). We simulate affective inference in a T-maze paradigm requiring context learning, followed by context reversal. The result is a deep (biologically plausible) agent that infers its affective state and reduces its uncertainty through internal action (i.e., optimises prior beliefs that underwrite confidence). Thus, we demonstrate the potential of active inference in providing a formal and computationally tractable account of the link between affect, (mental) action, and implicit meta-cognition.

Integrated World Modeling Theory (IWMT) Revisited

“…This prediction of prediction error minimization creates an interesting setup in which such organisms end up being surprised when they do not find themselves riding down steep enough gradients of prediction error (Joffily and Coricelli, 2013;Hesp et al, 2019). This is exactly what evolution would 'want' (Dawkins, 1996;Safron, 2019b), since there is no limit to how evolutionarily fit an organism can be, and so organisms ought to always seek opportunities for realizing value in new ways.…”

“…Much of the phenomenology of desire could potentially be explained as the prediction of attaining value, which then activates associated somatic and interoceptive concomitants of consummation, which are subjectively felt in body maps in those places most associated with value realization. If these sensations are accompanied by temporary net decreases in predicting the realization of homeostatic or reproductive value (Joffily and Coricelli, 2013;Hesp et al, 2019)-potentially mediated by opioid signaling (Leknes and Tracey, 2008;Berridge and Kringelbach, 2015)-this state would be likely to result in net negative affect accompanying these perceptions. In this way, the feeling of desire may be experienced as a kind of pain, with its particular characteristics depending on particular histories of associated experience.…”

“…How can motivation be maintained if all pleasures will ultimately be transformed back into kinds of pain? One explanation may potentially be found through predictive coding mechanisms, in that nervous systems will select for perception and memory that which changes the most, and so experiencing/remembering will likely to be dominated by the transitions between pleasure and pain, especially if accompanied by precipitous or punctuated alterations (Gopnik, 1998;Joffily and Coricelli, 2013;Safron, 2016;Hesp et al, 2019;Safron, 2019c). If consummation tends to rectify a situation more rapidly, and if seeking without finding results in a more gradual buildup of desire, then experience and memory for successfully enacted goals may have an overall pleasurable (and reinforcing) quality.…”

Dissolving the self

“…In just the same way that a hungry organism can act so as to harvest confirmatory evidence for the hypothesis "I am sated", hypotheses relating to higher-levels of the self-model geared towards control of outcomes on longer timescales act to constrain action in the present to bring downstream outcomes closer in line with the prior expectation. Overly precise priors driving action on a long timescale which are failing to be fulfilled, on this view, would be a persistent cause of suffering, due to the system consistently failing to meet (or align actions towards) the goal state (Hesp, Smith, Allen, Friston, & Ramstead, 2019). Under the model of psychedelic-induced ego-dissolution proposed, the high-precision highlevel priors geared towards control on multiple timescales cease to exert influence on the system due to the proposed lowering of precision of high-level priors under psychedelics.…”

Choosing a Markov blanket

“…As such, the most informative observation is the one that resolves the most amount of uncertainty, which means that agents may seek out the most entropic stimuli in order to derive the greatest amount of information. Finally, recent active inference formulations parameterize expected free energy (Hesp et al 2019); in this setting, agents have beliefs or preferences about how surprised they should typically be. These qualifications about the relationship between the entropy and active inference naturally account for the findings discussed by Fortier-Davy that humans tend to prefer medium-entropy rather than low-entropy stimuli.…”