Adaptive behavior requires the sensing of salient behavioral consequences which can act to modulate changes in neural connections linking sensory and motor structures. In previous work, we proposed that salient sensory events trigger neuronal value systems capable of modulating synaptic plasticity. Here, we investigate the capacity of value systems to modulate their own responses in the context of various conditioning tasks. To this end, we implement a modifiable value system incorporating anatomical and physiological properties within Darwin V, a neuronal model embedded in a mobile real world device. While exploring an environment containing stimulus objects, Darwin V's visual maps develop object-related neuronal responses. Phasic responses of a value system initially triggered only by object-"taste" (innate value) modulate changes in connections between visual and motor neurons, thus linking specific visual responses to appropriate motor outputs. Over time, Darwin V is able behaviorally to discriminate between "striped" objects with positive value (appetitive behavior) and objects with "blobs" with negative value (aversive behavior) based on vision alone. In parallel with modification of visuo-motor connections, value-dependent modification also occurs in connections from visual sensory maps to the value system itself. As a result, visual activity patterns become able directly to trigger value signals (acquired value). If acquired value is disabled, transfer of the value signal to stimuli preceding innately salient events does not occur, and behavioral responses due to aversive conditioning are subject to rapid extinction. If an auditory signal reliably precedes the visual appearance of an aversive object, Darwin V could be conditioned first to reject the object based on vision (primary conditioning), and subsequently based on sound alone (secondary conditioning). We compare the functional characteristics of value-dependent learning to formal notions of reinforcement learning. We suggest that plasticity in sensory afferents to value systems may provide a neurobiological basis for mediating the changing effects of saliency on adaptive behavioral responses.
