PROBABILISTICALLY REINFORCED CHOICE BEHAVIOR IN PIGEONS¹

Abstract

A single principle, "momentary maximizing", may account for much of a pigeon's steadystate behavior in both probability learning and concurrent variable interval experiments. The principle states that a pigeon tends to choose the alternative that momentarily has the higher probability of reinforcement. A successive discrimination procedure, which produced matching in an earlier experiment, produced here a tendency to maximize if training were adequately extended. Maximizing was produced also by other procedures, in which no reinforcing event was presented on some trials: one procedure did and two did not provide a bird with information about the availability of reinforcement on a key after an unreinforced response on the other key. The latter two procedures were analogous to colncurrent variable interval schedules in two respects: the reinforcement probability on one key increased while a bird responded on the other key; and they produced matching. But sequential statistics suggested that matching resulted from momentary maximizing. Depending on the procedure, the tendency to maximize produced different relative frequencies of pecking a key for a fixed relative frequency of reinforcement. Computer simulation of maximizing behavior in several concurrent variable interval schedules produced matching and sequential statistics similar to those produced by a real bird.The present sequence of experiments on two-choice behavior of pigeons reveals that a single principle may account for the apparent inconsistency of behavior produced by two kinds of experiments: discrete trial probability learning experiments and concurrent variable interval reinforcement schedules. This principle, "momentary maximizing", states that pigeons tend to choose the alternative that momentarily has the higher probability of reinforcement.Graf, Bullock, and Bitterman (1964) proximated the relative frequency with which that response was reinforced. These two experiments employed either a successive or a simultaneous visual discrimination, a center key and a correction procedure. Other visual problems as well as non-visual problems lead to maximizing (Bullock and Bitterman, 1962;Graf et al., 1964). In these experiments, maximizing was said to occur when the bird almost always chose the more frequently reinforced alternative. In short, Bitterman and his associates found that pigeons may match in visual problems but tend to maximize in nonvisual problems.Others (Catania, 1963;Herrnstein, 1961Herrnstein, , 1964Reynolds, 1963) have described the results of several concurrent variable interval experiments. The results show that pigeons produce a close match between the relative frequency of responding on a key and the relative frequency of reinforcement on it.To summarize, pigeons maximize in all but one class of discrete trial probability learning experiments; in other words, in all but certain visual discriminations. Yet they consistently match in concurrent variable interval experiments. The purpose of the following experiments was to account in ...

What are we doing when we translate from quantitative models?

“…In the present case, for instance, careful reading of the basic literature reveals that debate is unresolved about whether or when ratio-like schedules contribute to matching (Green, Rachlin, & Hanson, 1983;Herrnstein & Heyman, 1979;MacDonnall, 1988;Savastano & Fantino, 1994;Shimp, 1966;Shurtleff & Silberberg, 1990). Gaps in basic research limit the functional parallels that can be drawn when a laboratoryderived equation is fit to field data.…”

Information processing by pigeons (Columba livia): Incentive as information.

“…The fact that information theory provides an elegant description of how pigeons track rapidly changing statistical information in the serial reaction time task may clarify a possible mechanism underlying the molecular position, according to which pigeons generally can and do track continuous changes in reinforcement probability and may even do so with nearly optimal precision. Such a position is called momentary maximizing (Hinson & Staddon, 1983;Shimp, 1966Shimp, , 2004. In particular, if local reinforcement probability is viewed as local uncertainty, then processing local reinforcement probability is processing local statistical information.…”

Interresponse time as a factor in choice

“…Following reinforcement, the initial links were reinstated. If both terminal Shimp (1966) has shown that pigeons' choices of an alternative often vary as a function of the location of previous responses (left or right). The present study examined choice as a function of the time since the last choice response (interresponse time or IRT).…”