Evolution of an elemental theory of Pavlovian conditioning

Abstract

Kenneth Spence (1936, 1937) formalized a quantitative, elemental approach to association theory that has had a broad and dominating influence on learning theory for many years. A set of challenges to the basic approach has spurred the subsequent evolution of elemental theory in various ways. Four of the challenges and some resulting theoretical accommodations are described in the context of Pavlovian conditioning. The evolution involves departures from important specifics of Spence's theory, but is viewed as demonstrating the utility of the basic, elemental approach that is one of his legacies.

Summation of reinforcement rates when conditioned stimuli are presented in compound.

“…While the findings reported here are problematic for the Rescorla-Wagner (1972) model, they may appear less so for numerous other associative models (Harris, 2006;McLaren & Mackintosh, 2002;Pearce, 1994;Wagner, 2003;Wagner & Brandon, 2001), even though these models use the same learning rule defined in Equation 1. These more recent models all contain elaborated descriptions of how stimuli are represented within the learning mechanism, and in particular they invoke nonlinear rules to account for generalisation of V between individual CSs and their compounds.…”

“…In each case, the size of the decrement, and thus the difference between V AB and [V A + V B ], can vary depending on factors such as the salience of the CSs versus the context, and the degree of similarity between the CSs. Indeed, in one of these models (Wagner, 2003;Wagner & Brandon, 2001), the amount of generalisation is set by a parameter that permits anything from complete summation, V AB = V A + V B , to a complete failure of generalisation between CSs and their compound, V AB = 0. As such, these more recent associative models tend to avoid making empirically-testable predictions about the amount of summation that will occur between CSs.…”

Response rates track the history of reinforcement times.

“…The findings reported here are problematic for real-time associative models that attribute changes in responding across the course of a CS to differential conditioning of temporally distributed CS elements (Desmond & Moore, 1988;Sutton & Barto, 1981, 1990Wagner & Brandon, 2001). The type of competitive learning rule (Rescorla & Wagner, 1972) typically used by these models distributes associative strength among the elements of the CS in proportion to their rate of reinforcement.…”

“…An alternative approach maintains the central features of associative processes, but attributes the evidence for response timing to a representation of the CS that is distributed across time. The most popular version of this approach, dating back to Pavlov (1927), assumes that the CS is represented by multiple elements with different temporal characteristics (e.g., Desmond & Moore, 1988;Sutton & Barto, 1981, 1990Wagner & Brandon, 2001). Each element serves as a "micro-CS", and the elements that are most active at the time of the US gain the majority of associative strength according to a competitive learning rule like that proposed by Rescorla and Wagner (1972).…”

Response rate and reinforcement rate in Pavlovian conditioning.

“…Rescorla and Wagner (1972) greatly extended the explanatory scope of this simple description by proposing that learning on any trial is proportional to the difference between  and the sum of what has been learned about all CSs present on the current trial (V). This simple error-correction rule, or an operation that is functionally equivalent, has been incorporated into numerous more elaborate models of learning (Desmond & Moore, 1988;Grossberg & Schmajuk, 1989;Harris, 2006;McLaren & Mackintosh, 2000;Pearce, 1994;Sutton & Barto, 1981;Wagner, 1981;Wagner & Brandon, 2001). …”