A single principle, "momentary maximizing", may account for much of a pigeon's steadystate behavior in both probability learning and concurrent variable interval experiments. The principle states that a pigeon tends to choose the alternative that momentarily has the higher probability of reinforcement. A successive discrimination procedure, which produced matching in an earlier experiment, produced here a tendency to maximize if training were adequately extended. Maximizing was produced also by other procedures, in which no reinforcing event was presented on some trials: one procedure did and two did not provide a bird with information about the availability of reinforcement on a key after an unreinforced response on the other key. The latter two procedures were analogous to colncurrent variable interval schedules in two respects: the reinforcement probability on one key increased while a bird responded on the other key; and they produced matching. But sequential statistics suggested that matching resulted from momentary maximizing. Depending on the procedure, the tendency to maximize produced different relative frequencies of pecking a key for a fixed relative frequency of reinforcement. Computer simulation of maximizing behavior in several concurrent variable interval schedules produced matching and sequential statistics similar to those produced by a real bird.The present sequence of experiments on two-choice behavior of pigeons reveals that a single principle may account for the apparent inconsistency of behavior produced by two kinds of experiments: discrete trial probability learning experiments and concurrent variable interval reinforcement schedules. This principle, "momentary maximizing", states that pigeons tend to choose the alternative that momentarily has the higher probability of reinforcement.Graf, Bullock, and Bitterman (1964) proximated the relative frequency with which that response was reinforced. These two experiments employed either a successive or a simultaneous visual discrimination, a center key and a correction procedure. Other visual problems as well as non-visual problems lead to maximizing (Bullock and Bitterman, 1962;Graf et al., 1964). In these experiments, maximizing was said to occur when the bird almost always chose the more frequently reinforced alternative. In short, Bitterman and his associates found that pigeons may match in visual problems but tend to maximize in nonvisual problems.Others (Catania, 1963;Herrnstein, 1961Herrnstein, , 1964Reynolds, 1963) have described the results of several concurrent variable interval experiments. The results show that pigeons produce a close match between the relative frequency of responding on a key and the relative frequency of reinforcement on it.To summarize, pigeons maximize in all but one class of discrete trial probability learning experiments; in other words, in all but certain visual discriminations. Yet they consistently match in concurrent variable interval experiments. The purpose of the following experiments was to account in ...

Expected Utility Theory successfully predicts the steady-state relative frequencies of the two alternatives in two-key, discrete-trial probability-learning experiments, in concurrent variable interval schedules, and in concurrent variable interval schedules modified to study magnitude and delay of reinforcement and conditioned reinforcement. It also successfully predicts the relative frequencies of interresponse times in one-key variable interval schedules. Different interresponse times between pecks on a single key and pecks on different keys are treated as different instances of concurrent operants that have basically the same functional relationships with reinforcement variables. The theory states that an 5" chooses whichever alternative momentarily has the greatest weighted probability of reinforcement.

Three pigeons performed on two-component multiple variable-interval variable-interval schedules of reinforcement. There were two independent variables: component duration and the relative frequency of reinforcement in a component. The component duration, which was always the same in both components, was varied over experimental conditions from 2 to 180 sec. Over these conditions, the relative frequency of reinforcement in a component was either 0.2 or 0.8 (+0.03). As the component duration was shortened, the relative frequency of responding in a component approached a value equal to the relative frequency of reinforcement in that component. When the relative frequency of reinforcement was varied over conditions in which the component duration was fixed at 5 sec, the relative frequency of responding in a component closely approximated the relative frequency of reinforcement in that component. Fig. 3a in Reynolds, 1963, and Fig. 2

The relative magnitude and relative frequency of reinforcement for two concurrent interresponse times (1.5 to 2.5 sec and 3.5 to 4.5 sec) were simultaneously varied in an experiment in which pigeons obtained grain by pecking on a single key. Visual discriminative stimuli accompanied the two time intervals in which reinforcements were arranged by a one-minute variable-interval schedule. The resulting interresponse times of each of three pigeons fell into two groups; "short" (1.0 to 2.5 sec) and "long" (3.0 to 4.5 sec). Steady-state relative frequencies of these interresponse times were orderly functions of both reinforcement variables. The combined effects of both independent variables were well summarized by a linear function of one variable, relative access to food. Unlike corresponding two-key concurrent variable-interval schedules, the present schedule did not produce an equality between the relative frequency of an operant and either the relative magnitude or the relative frequency of reinforcement of that operant. A tentative account is provided for this difference between one-key and two-key functions.A basic problem in the experimental analysis of behavior is to determine the functions relating relative frequencies of concurrent operants to the relative magnitudes and relative frequencies of reinforcement for those operants. Here the problem was attacked with an experiment using variable-interval reinforcement and pigeons. Two ment for pecks on that key (Herrnstein, 1961;Catania, 1963; Reynolds, 1963;Neuringer, 1967).The second approach, that in which an operant is a class of IRTs, has been less successful. Anger (1954) asked if the relative frequencies of occurrence of the different IRTs in a variable-interval schedule depended on the corresponding relative frequencies of reinforcement. To answer this question he devised a "synthetic" variable-interval schedule; a concurrent schedule of reinforcement for different IRTs. His results suggested that this dependency may exist sometimes but also that sometimes it may seem too small to account for the development of normal variable-interval behavior. Specifically, Anger showed that a decrease in reinforcement frequency for one band of IRTs decreased the frequency of occurrence of responses terminating in that band, but that a corresponding increase in reinforcement frequency for another band had no effect. Thus, it is not clear how much the relative frequencies of IRTs depend on the relative frequencies of reinforcement for them. But in any case, this dependency is not the same as the matching function in two-key concurrent variable-interval schedules. Before now this second approach has not been used with reinforcement magnitude, although Hendry (1962) has shown in a different context 525 1968, 11,[525][526][527][528][529][530][531][532][533][534][535] NUMBER 5 (SEPTEMBER)

Temporal patterns of key pecking by pigeons were shaped by a schedule in which the delivery of food was contingent upon a measure of the overall extent to which the temporal pattern of behavior within a 5-sec trial conformed to a required pattern. This pattern approximated a constant rate of change in the rate of key pecking throughout the 5-sec trial. In comparison with behavior maintained by a classical fixed-interval 5-sec schedule, the new schedule controlled a better approximation to a "scallop" within individual trials and greatly reduced intersubject variability. These results are consistent with the view that the delivery of a reinforcer after a behavioral pattern a few seconds in duration may strengthen the entire pattern as a unit, or operant. The response topography contiguous with reinforcement may be a negligible fraction of the strengthened operant. One implication of this view is that mean response rate for such brief responses as key pecks and lever presses is a byproduct of whatever patterns are strengthened, and generally will not reveal fundamental controlling relationships, whenever a reinforcer is not contiguous with all the behavior on which it is contingent.