Dream to Explore: 5-HT2a as Adaptive Temperature Parameter for Sophisticated Affective Inference

Abstract

Relative to other neuromodulators, serotonin (5-HT) has received far less attention in machine learning and active inference. We will review prior work interpreting 5-HT1a signaling as an uncertainty parameter with opponency to dopamine. We will then discuss how 5-HT2a receptors may promote more exploratory policy selection by enhancing imaginative planning (as sophisticated affective inference). Finally, we will briefly comment on how qualitatively different effects may be observed across low and high levels of 5-HT2a signaling, where the latter may help agents to change self-adversarial policies and break free of maladaptive absorbing states in POMDPs.

Value Cores for Inner and Outer Alignment: Simulating Personality Formation via Iterated Policy Selection and Preference Learning with Self-World Modeling Active Inference Agents

“…We will specifically focus on diffuse neuromodulatory systems of the brain controlling dopamine (DA) and serotonin (5-HT) signaling, which are involved in a wide variety of cognitive, affective, and motoric functions and in developing intelligent (and agentic) systems. While midbrain DA neurons elicit reward-related behavior, 5-HT receptors have been suggested to often operate in an opponent fashion, including with respect to the mediation of either "passive coping" or "active coping" strategies in the face of uncertainty [32,33]. Within FEP-AI, neurons with D1 receptors are suggested to compute free energy expected under a given policy, with phasic DA release associated with reward prediction errors [34].…”

“…Most research into biologically-inspired ML (and FEP-AI) parameters have focused on the roles of DA. However, we will also consider 5-HT, as well as its potentially heterogeneous effects with respect to different receptor classes [32]. In autonomous systems inspired by serotonin-like parameterization, uncertainty is used to bias control in favor of inner-loop (or more model-based) decision-making processes, relative to outer-loop (or more model-free and 'reflexive' or 'impulsive') dynamics or amortised inference [36].…”

Towards Understanding Persons and Their Personalities with Cybernetic Big 5 Theory and the Free Energy Principle and Active Inference (FEP-AI) Framework

“…Notably with respect to computational psychiatry (and also ethology, these are the neurotransmitter systems agonized by SSRIs for depression and anxiety (and also dominance within social hierarchies). However, extreme levels of serotonergic signaling have been associated with the "relaxation" of beliefs and greater exploration in FEP-AI [67]- [69]. While CB5T has previously emphasized serotonin's potential role in modulating Stability, potential roles of this system for enhancing Plasticity is a promising direction for future research.…”

A dual-receptor model of serotonergic psychedelics

“…In passive coping the animal maintains the current behavioral policy but modulates affect in response to manageable levels of stress. In contrast, active coping is engaged when the level of stress is significant enough to require the instantiation of novel and divergent internal models, beliefs, or behavioral policies (Brouwer & Carhart-Harris, 2021; Safron & Sheikhbahaee, 2021). This bi-modal behavioral distribution can be seen in a simplified form in serotonin-mediated rodent responses to stress from predatory threats, where a low-level threat initiates freezing behavior while a high-level threat initiates fleeing behavior (Seo et al, 2019).…”