After a relatively small amount of training, instrumental behavior is thought to be an action under the control of the motivational status of its goal or reinforcer. After more extended training, behavior can become habitual and insensitive to changes in reinforcer value. Recently, instrumental responding has been shown to weaken when tested outside of the training context. The present experiments compared the sensitivity of instrumental responding in rats to a context switch after training procedures that might differentially generate actions or habits. In Experiment 1, lever pressing was decremented in a new context after either short, medium, or long periods of training on either random-ratio or yoked random-interval reinforcement schedules. Experiment 2 found that more minimally-trained responding was also sensitive to a context switch. Moreover, Experiment 3 showed that when the goal-directed component of responding was removed by devaluing the reinforcer, the residual responding that remained was still sensitive to the change of context. Goal-directed responding, in contrast, transferred across contexts. Experiment 4 then found that after extensive training, a habit that was insensitive to reinforcer devaluation was still decremented by a context switch. Overall, the results suggest that a context switch primarily influences instrumental habit rather than action. In addition, even a response that has received relatively minimal training may have a habit component that is insensitive to reinforcer devaluation but sensitive to the effects of a context switch.

An occasion setter is a stimulus that modulates the ability of another stimulus to control behavior. A rich history of experimental investigation has identified several important properties that define occasion setters and the conditions that give rise to occasion setting. In this paper, we first consider the basic hallmarks of occasion setting in Pavlovian conditioning. We then review research that has examined the mechanisms underlying the crucial role of context in Pavlovian and instrumental extinction. In Pavlovian extinction, evidence suggests that the extinction context can function as a negative occasion setter whose role is to disambiguate the current meaning of the conditioned stimulus; the conditioning context can also function as a positive occasion setter. In operant extinction, in contrast, the extinction context may directly inhibit the response, and the conditioning context can directly excite it. We outline and discuss the key results supporting these distinctions.

Pigeons' pecks produced grain under progressive ratio (PR) schedules, whose response requirements increased systematically within sessions. Experiment 1 compared arithmetic (AP) and geometric (GP) progressions. Response rates increased as a function of the component ratio requirement, then decreased linearly (AP) or asymptotically (GP). Experiment 2 found the linear decrease in AP rates to be relatively independent of step size. Experiment 3 showed pausing to be controlled by the prior component length, which predicted the differences between PR and regressive ratio schedules found in Experiment 4. When the longest component ratios were signaled by different key colors, rates at moderate ratios increased, demonstrating control by forthcoming context. Models for response rate and pause duration described performance on AP schedules; GP schedules required an additional parameter representing the contextual reinforcement. Keywordscontext; fixed ratio; mathematical model; post-reinforcement pause; progressive ratio; reinforcement schedules One of B. F. Skinner's bequests was his emphasis on the singular importance of the schedule of reinforcement, the contingencies that the experimenter or environment arranges for the presentation of events to an organism. The events could be biologically significant or insignificant, positive or negative. Nature, not Skinner, was the first to arrange reinforcement schedules, and many scientists-Pavlov and Thorndike the exemplars-used them to good effect before Skinner. But Skinner, along with his students (e.g., Ferster & Skinner, 1957) made them an end, not means, of inquiry. One of the most fundamental type of schedule arranges reinforcement contingent upon the completion of a fixed number of responses, the Fixed Ratio (FR) schedule. If the ratio is not fixed, but varies widely from one reinforcer to the next, the schedule is called Variable Ratio (VR). Ratio schedules engender high rates of responding, more or less uniform under VR, consisting of a post-reinforcement pause or break followed by a run of high rate responding under FR.It is possible to schedule ratio reinforcement intermediate between these extremes of variability. Under Progressive Ratio (PR) schedules the requirements for reinforcement are increased systematically, usually after each reinforcer. Since their introduction by Hodos (1961), PR schedules have been used increasingly to assess reinforcement strength, an assay of special importance to behavioral pharmacologists (e.g., Ping-Teng, Lee, Konz, Richardson, Correspond with: Peter Killeen, Department of Psychology, Box 871104, Arizona State University, Tempe, AZ 85287-1104, Killeen@asu.edu, FAX: (480) Voice: (480) 967-0560. Publisher's Disclaimer: The following manuscript is the final accepted manuscript. It has not been subjected to the final copyediting, fact-checking, and proofreading required for formal publication. It is not the definitive, publisher-authenticated version. The American Psychological Association and its Council of Editors discl...

Goal-directed actions are instrumental behaviors whose performance depends on the organism’s knowledge of the reinforcing outcome’s value. In contrast, habits are instrumental behaviors that are insensitive to the outcome’s current value. Although habits in everyday life are typically controlled by stimuli that occasion them, most research has studied habits using free-operant procedures in which no discrete stimuli are present to occasion the response. We therefore studied habit learning when rats were reinforced for lever pressing on a random interval 30-s schedule in the presence of a discriminative stimulus (S), but not in its absence. In Experiment 1, devaluing the reinforcer with taste aversion conditioning weakened instrumental responding in a 30-s S after 4, 22, and 66 sessions of instrumental training. Even extensive practice thus produced goal-directed action, not habit. Experiments 2 and 3 contrastingly found habit when the duration of S was increased from 30 s to 8 min. Experiment 4 then found habit with the 30-s S when it always contained a reinforcer; goal-directed action was maintained when reinforcers were earned at the same rate, but occurred in only 50% of Ss (as in the previous experiments). The results challenge the view that habits are an inevitable consequence of repeated reinforcement (as in the Law of Effect), and instead suggest that discriminated habits develop when the reinforcer becomes predictable. Under those conditions, organisms may pay less attention to their behavior, much as they pay less attention to signals associated with predicted reinforcers in Pavlovian conditioning.

No abstract