Plasticity in Value Systems and its Role in Adaptive Behavior

Abstract

Adaptive behavior requires the sensing of salient behavioral consequences which can act to modulate changes in neural connections linking sensory and motor structures. In previous work, we proposed that salient sensory events trigger neuronal value systems capable of modulating synaptic plasticity. Here, we investigate the capacity of value systems to modulate their own responses in the context of various conditioning tasks. To this end, we implement a modifiable value system incorporating anatomical and physiological properties within Darwin V, a neuronal model embedded in a mobile real world device. While exploring an environment containing stimulus objects, Darwin V's visual maps develop object-related neuronal responses. Phasic responses of a value system initially triggered only by object-"taste" (innate value) modulate changes in connections between visual and motor neurons, thus linking specific visual responses to appropriate motor outputs. Over time, Darwin V is able behaviorally to discriminate between "striped" objects with positive value (appetitive behavior) and objects with "blobs" with negative value (aversive behavior) based on vision alone. In parallel with modification of visuo-motor connections, value-dependent modification also occurs in connections from visual sensory maps to the value system itself. As a result, visual activity patterns become able directly to trigger value signals (acquired value). If acquired value is disabled, transfer of the value signal to stimuli preceding innately salient events does not occur, and behavioral responses due to aversive conditioning are subject to rapid extinction. If an auditory signal reliably precedes the visual appearance of an aversive object, Darwin V could be conditioned first to reject the object based on vision (primary conditioning), and subsequently based on sound alone (secondary conditioning). We compare the functional characteristics of value-dependent learning to formal notions of reinforcement learning. We suggest that plasticity in sensory afferents to value systems may provide a neurobiological basis for mediating the changing effects of saliency on adaptive behavioral responses.

Symbolic Models and Emergent Models: A Review

“…The examples include McClelland et al [143], Weng et al [30], Elman et al [34], Sprons et al [144], Wiemer [145], Roelfsema & VanOoyen [146], Sit & Miikkulainen [147], Hinton [148], and Weng et al [149] where the representations are mostly emergent although some of these models do not exactly satisfy our strict Definition 5 for a fully emergent representation. The epigenetic development (ED) network [114] (later called DN) further suggests that the meaning of any neuron or any brain area cannot be precisely described in a human language.…”

Modeling dopamine and serotonin systems in a visual recognition network

“…Almassy et al 1998 [1], further refined in Sporns et al 2000 [24], proposed a neuromorphic architecture for learning primary and seconding conditioning that tend to avoid actions that lead to punishments and adopt actions that lead to reward. Cox & Krichmar 2007 [4] experimented with a neuromorphic architecture that integrates three types of neurotransmitters, 5-HT, DA and Ach with Ach for increased attention efforts.…”

Task Transfer by a Developmental Robot

“…For example, in [8] and [9] a robot functioned upon a set of percepts and behaviors that were defined in advance. In [10], a mobile robot learned to associate the visual sensations of predefined classes (blob and stripe patterns on cubes) with predefined action sequences. These studies have greatly advanced computational understanding of animal learning.…”

“…According to the operation of the architecture, the action is executed. This developmental learning process is denoted as (10) This is the second phase of communicative learning: learning using acquired sensorimotor skills. This type of learning is typically more efficient than exploration since the desired action is hinted directly by the human teacher, instead of based on trying all possible actions using, e.g., Boltzmann exploration.…”