Model of the interactions between neuromodulators and prefrontal cortex during a resource allocation task

Abstract

Neuromodulators such as dopamine (DA), serotonin (5-HT), and acetylcholine (ACh) are crucial to the representations of reward, cost, and attention respectively. Recent experiments suggest that the reward and cost of actions are also partially represented in orbitofrontal and anterior cingulate cortices in that order. Previous models of action selection with neuromodulatory systems have not extensively considered prefrontal contributions to action selection. Here, we extend these models of action selection to include prefrontal structures in a resource allocation task. The model adapts to its environment, modulating its aggressiveness based on its successes. Selective lesions demonstrate how neuromodulatory and prefrontal areas drive learning and performance of strategy selection.

A neurorobotic platform to test the influence of neuromodulatory signaling on anxious and curious behavior

“…On two different robot platforms, they demonstrated that their model could deal with both expected and unexpected uncertainties in the real world. Our group has recently investigated the possible role of multiple neuromodulators in a resource allocation task (Chelian et al, 2012), and reversal learning on an autonomous robot (Oros and Krichmar, 2012). …”

“…For example, the medial prefrontal cortex (mPFC) can control the stress response by its interaction with the raphe nucleus, the main source of 5-HT in the central nervous system (Jasinska et al, 2012), and the orbitofrontal cortex (OFC) may exert control on the DA reward system (Frank and Claus, 2006). Empirical evidence and theoretical modeling have suggested that the mPFC, the anterior cingulate cortex, and the OFC control decision-making in the face of reward-cost tradeoffs (Rudebeck et al, 2006; Rushworth et al, 2007; Chelian et al, 2012). That is, the OFC's interaction with the DA system is monitoring the expected reward of an action, and the mPFC's interaction with the 5-HT system is monitoring the expected cost of an action (Zaldivar et al, 2010; Asher et al, 2012).…”

The neural basis of decision-making during sensemaking: Implications for human-system interaction

“…(As an example, if asked to repeatedly predict the outcome of a tail-heavy coin, humans often alternate between the heads and tails while the rational decision would be to always predict tails.) Here we emulate characteristics of human performance in an analog resource allocation task by adapting a recently developed neural network [42]. Both subjects and the neural network demonstrated a tendency towards probability matching, especially after trials with lower than expected reward outcomes.…”

“…To simulate and predict subject behavior, we adapted the recent neural network of [42]. This uses neurally plausible temporal difference dynamics to modulate its aggressiveness in response to feedback from the environment.…”

“…The model was also applied to a prognostic foraging task wherein agents had to choose where to collect information to better predict an attack [44]. In that work, information gain was equated with reward thus demonstrating the ability of the original model of [42] to be applied to different problem domains.…”