This document is copyrighted by the American Psychological Association or one of its allied publishers.This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Although responses are sometimes easy to predict, at other times responding seems highly variable, unpredictable, or even random. The inability to predict is generally attributed to ignorance of controlling variables, but this article is a review of research showing that the highest levels of behavioral variability may result from identifiable reinforcers contingent on such variability. That is, variability is an operant. Discriminative stimuli and reinforcers control it, resulting in low or high variability, depending on the contingencies. Schedule-of-reinforcement effects are orderly, and choosing to vary or repeat is lawfully governed by relative reinforcement frequencies. The operant nature of variability has important implications. For example, learning, exploring, creating, and problem solving may partly depend on it. Abnormal levels of variability, including those found in psychopathologies such as autism, depression, and attention deficit hyperactivity disorder, may be modified through reinforcement. Operant variability may also help to explain some of the unique attributes of voluntary action.

Experimental psychologists generally maintain that people cannot behave randomly. The present experiment asked students to generate random sequences of two numbers on the keyboard of a computer terminal. At first, all subjects' sequences differed significantly from random, thereby replicating the findings of the literature. But when given feedback from 5 or 10 statistical descriptors, the subjects learned to generate sequences that were indistinguishable, according to these statistics, from computergenerated random numbers. Randomlike behavior can therefore be learned.When asked to behave randomly, people generally fail (for reviews, see Tune, 1964aTune, , 1964bWagenaar, 1972). An experiment, by Bakan ( 1960) is a good illustration of the methods used in tests of "random" behavior. Seventy undergraduates were asked "to produce a series of 'heads' and 'tails' such as they might expect to occur if an unbiased coin were tossed in an unbiased manner for a total of 300 independent tosses." The subjects filled in "H" or "T" boxes on a form. Analysis of the frequencies of runs and triplets showed that responses differed from those expected by chance.The failure of human subjects to behave randomly is a robust finding. The number of alternative responses have varied from 2,asintheworkofBakan

Behavior is sometimes insensitive, and sometimes extremely sensitive, to changes in reinforcement magnitude. The present work attempted to analyze this disparity by comparing, in a single experimental situation, a pigeon's choices with its response rates. Whereas choices varied directly with reinforcement duration, rates of responding were comparatively insensitive to duration changes. These results suggest that the effect of reinforcement magnitude on responding partly depends upon the extent to which responding influences the amount of reinforcement.

Walter Mischel studied self-control in preschool children in the following manner: if the child waited for an interval to end, he or she received the more preferred of two reinforcers; if the child responded to terminate the interval by ringing a bell, the less preferred reinforcer was given. We used an analogous procedure to study self-control in pigeons: if the bird waited for a trial to end, it received the more preferred reinforcer; if the bird terminated the trial by pecking a key, the less preferred reinforcer was given. We explored the effects on self-control of a number of variables analogous to those studied by Mischel and co-workers, e.g., presence versus absence of reinforcers, of alternative responses, and of stimuli during the wait interval; prior experience of the subjects; and test paradigm. The results obtained with pigeons paralleled the results obtained by Mischel with human children.

Some studies have found that extinction leaves response structures unaltered; others have found that response variability is increased. Responding by Long-Evans rats was extinguished after 3 schedules. In one, reinforcement depended on repetitions of a particular response sequence across 3 operanda. In another, sequences were reinforced only if they varied. In the third, reinforcement was yoked: not contingent upon repetitions or variations. In all cases, rare sequences increased during extinctionvariability increased-but the ordering of sequence probabilities was generally unchanged, the most common sequences during reinforcement continuing to be most frequent in extinction. The rats' combination of generally doing what worked before but occasionally doing something very different may maximize the possibility of reinforcement from a previously bountiful source while providing necessary variations for new learning.

No abstract

Previous research has demonstrated that behavioral variability can be modified by reinforcers contingent on it, but there has been no convincing evidence of discriminative stimulus control over such variability. We therefore rewarded 20 rats for variable response sequences in the presence of one stimulus and provided equal rewards independently of sequence variability in the presence of a second stimulus. We found that sequence variability was significantly higher during the first stimulus than during the second, with the greatest difference occurring immediately following onset of the stimuli. Removing the discriminative stimuli caused levels of variability to converge. These experiments provide strong evidence that behavioral variability can be controlled by discriminative stimuli, which may be important for general theories of operant behavior and their applications.Although variability is characteristic of all physical phenomena, a particularly adaptive form may be seen in the behavior of complex organisms. We refer to variability that is shaped and influenced by consequences-that is, operant or instrumental variability. Evidence for operant variability was obtained when rats or pigeons were reinforced for variable sequences of left (L) and right (R) responses-for example, LLRL, RRLR, and LLLL (see, e.g., Neuringer & Huntley, 1992;Page & Neuringer, 1985). For reinforcement, a sequence was required to differ from each of the previous three sequences. For example, given the just enumerated three sequences, ifthe current sequence was RLRL, it led to reinforcement, but ifit was RRLRa repetition-then reinforcement was withheld. When variability was compared to that under a yoked control procedure-in which identical reinforcement was provided but without regard to sequence variability-a consistent finding was that variability was higher when reinforced than when not (see also Blough, 1966;Bryant & Church, 1974;Holman, Goetz, & Baer, 1977;Machado, 1989Machado, , 1992Machado, , 1993Manabe, Staddon, & Cleaveland, 1997; Morgan & Neuringer, 1990;Pryor, Haag, & O'Reilly, 1969; but for different results see Schwartz, 1982aSchwartz, , 1988. This sensitivity to contingencies of reinforcement is one defining characteristic of operant behavior.Further support for the operant nature ofvariability came from the observation that pigeons' choices to vary or repeat followed the same matching function that describes This research was supported in part by grants from the National Science Foundation to A.N. Portions of these data were reported by J.D. in a Reed College undergraduate thesis. We thank Gene Olson and Chris Deiss for excellent animal care and programming assistance and Armando Machado for a helpful reading ofthe manuscript. Correspondence concerning this article should be addressed to A. Neuringer, Psychology Department, Reed College, Portland, OR 97202 (e-mail: allen_neuringer@ reed.edu).-Accepted by previous editor, Robert A. Rescorla simpler choices. Specifically, the probability of varying versus repeating was c...