Intertrial unconditioned stimuli differentially impact trace conditioning

Abstract

Three experiments assessed how appetitive conditioning in rats changes over the duration of a trace conditioned stimulus (CS) when unsignaled unconditioned stimuli (USs) are introduced into the intertrial interval. In Experiment 1, a target US occurred at a fixed time either shortly before (embedded), shortly after (trace), or at the same time (delay) as the offset of a 120-s CS. During the CS, responding was most suppressed by intertrial USs in the trace group, less so in the delay group, and least in the embedded group. Unreinforced probe trials revealed a bellshaped curve centered on the normal US arrival time during the trace interval, suggesting that temporally specific learning occurred both with and without intertrial USs. Experiments 2a and 2b confirmed that the bulk of the trace CS became inhibitory when intertrial USs were scheduled, as measured by summation and retardation tests, even though CS offset evoked a temporally precise conditioned response. Thus, an inhibitory CS may give rise to new stimuli specifically linked to its termination, which are excitatory. A modification to the microstimulus temporal difference model is offered to account for the data.

Altered balance of excitatory and inhibitory learning in a genetically modified mouse model of glutamatergic dysfunction relevant to schizophrenia

“…During the trace interval, however, WT mice did make anticipatory magazine responses suggesting that the CS trace memory, or the offset of the CS, rather than the CS itself, was a signal for the occurrence of food. The ability of a trace conditioned stimulus to become a conditioned inhibitor is not unprecedented 28 – 31 . Furthermore, it suggests that the temporal discontiguity in the trace conditioning procedure may lead to not only a weakening of excitatory associative learning, but also an active strengthening of inhibitory learning.…”

Delay of reinforcement versus rate of reinforcement in Pavlovian conditioning.

“…Alternatively, if a componential view of CS representations is assumed such that a stimulus consists of a series of temporally ordered microstimuli (e.g., Ludvig et al, 2012; Sutton & Barto, 1981, 1990; Vogel et al, 2003), it is possible that the continued CS exposure after the US had no effect because the nature of the CS representation during those periods was different to its representation prior to reinforcement. Temporal difference learning models have appealed to changes in the nature of stimulus representations within a trial in order to explain various aspects of timing behavior (e.g., Williams, Todd, Chubala, & Ludvig, 2017). In the case of the results in Experiment 3, there would have had to be little or no generalization of learning between elements of the CS representation that were processed prior to reinforcement and after reinforcement.…”

From eye-blinks to state construction: Diagnostic benchmarks for online representation learning

“…On the question of representation or agent state, the answer is less clear. TD-models can generate predictions consistent with the animal data, but only if the state representation fills the gap between the CS and US in the right way ( Ludvig et al, 2009 , 2012 ; Williams et al, 2017 ). A flag indicating the CS just happened, called the presence representation , will not induce predictions that increase over time, and a clock is not plausible given the range of timescales, the presence of other non-relevant distracting signals, and the massive number of predictive relationships an agent must learn in its lifetime 3 ( Gallistel & King, 2011 ).…”

“…A flag indicating the CS just happened, called the presence representation , will not induce predictions that increase over time, and a clock is not plausible given the range of timescales, the presence of other non-relevant distracting signals, and the massive number of predictive relationships an agent must learn in its lifetime 3 ( Gallistel & King, 2011 ). Hand-designed temporal representations do reproduce the animal data well ( Ludvig et al, 2008 , 2009 , 2012 ; Williams et al, 2017 ), but their generality remains unclear. Ideally, the learning system could discover for itself how to represent different stimuli over-time in a way that (1) is useful across a variety of prediction tasks, and (2) requires computation and storage independent of the size of the trace interval.…”