Discrete-trial probability learning in rats: Effects of local contingencies of reinforcement

Hiraoka, Kyoichi

doi:10.3758/bf03199978

Cited by 7 publications

(9 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Brunswik (1939) first implemented the basic procedure in an elevated T-maze, while more recently reinforcement learning with multiple alternatives has been studied in the nine-hole box (Bari et al, 2010) and operant lever chambers (Hiraoka, 1984). In all of these task variants, rodents learn to preferentially select the more profitable response option as the number of trials increases.…”

Section: Reinforcement Learning Tasksmentioning

confidence: 99%

Measuring reinforcement learning and motivation constructs in experimental animals: Relevance to the negative symptoms of schizophrenia

Markou

Salamone

Bussey

et al. 2013

Neuroscience & Biobehavioral Reviews

View full text Add to dashboard Cite

The present review article summarizes and expands upon the discussions that were initiated during a meeting of the Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS; http://cntrics.ucdavis.edu). A major goal of the CNTRICS meeting was to identify experimental procedures and measures that can be used in laboratory animals to assess psychological constructs that are related to the psychopathology of schizophrenia. The issues discussed in this review reflect the deliberations of the Motivation Working Group of the CNTRICS meeting, which included most of the authors of this article as well as additional participants. After receiving task nominations from the general research community, this working group was asked to identify experimental procedures in laboratory animals that can assess aspects of reinforcement learning and motivation that may be relevant for research on the negative symptoms of schizophrenia, as well as other disorders characterized by deficits in reinforcement learning and motivation. The tasks described here that assess reinforcement learning are the Autoshaping Task, Probabilistic Reward Learning Tasks, and the Response Bias Probabilistic Reward Task. The tasks described here that assess motivation are Outcome Devaluation and Contingency Degradation Tasks and Effort-Based Tasks. In addition to describing such methods and procedures, the present article provides a working vocabulary for research and theory in this field, as well as an industry perspective about how such tasks may be used in drug discovery. It is hoped that this review can aid investigators who are conducting research in this complex area, promote translational studies by highlighting shared research goals and fostering a common vocabulary across basic and clinical fields, and facilitate the development of medications for the treatment of symptoms mediated by reinforcement learning and motivational deficits.

show abstract

Section: Reinforcement Learning Tasksmentioning

confidence: 99%

Measuring reinforcement learning and motivation constructs in experimental animals: Relevance to the negative symptoms of schizophrenia

Markou

Salamone

Bussey

et al. 2013

Neuroscience & Biobehavioral Reviews

View full text Add to dashboard Cite

show abstract

“…The apparent implication of Zeiler's result is that the overall reinforcement probability controlled behavior even when the local reinforcement contingencies were discriminated, by providing a response bias that competed with the control by the local contingencies. The results of Hiraoka (1984) and Zeiler (1987) suggest that control by local reinforcement contingencies and by molar reinforcement contingencies are in competition. The issue posed is how such joint control is to be incorporated into a theory of choice behavior.…”

mentioning

confidence: 92%

“…When local reinforcement contingencies have been pitted in opposition to molar reinforcement contingencies, separate roles appear to be played by both. Hiraoka (1984) trained rats on a discrete-trial choice procedure in which the overall probabilities of reinforcement for responses to two levers were .61 versus .39, which, according to the matching law, should produce exclusive preference for the lever with the higher probability (because the probability schedules were equivalent to a concurrent random-ratio random-ratio schedule; cf. Herrnstein & Loveland, 1975).…”

mentioning

confidence: 99%

“…Such control presumably was based on memory of the events from the preceding trial, which should become less effective with longer times since the preceding trial. After the effects of ITI were established in Experiment 1, Experiment 2 presented a choice procedure in which the two alternatives differed in their overall reinforcement probability but with differences in local probability of reinforcement still in effect, much like the procedure used by Hiraoka (1984). The issue was then how choice behavior was affected by variations in the ITI.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Choice as a Function of Local Versus Molar Reinforcement Contingencies

Williams¹

1991

J Exper Analysis Behavior

View full text Add to dashboard Cite

Rats were trained on a discrete-trial probability learning task. In Experiment 1, the molar reinforcement probabilities for the two response alternatives were equal, and the local contingencies of reinforcement differentially reinforced a win-stay, lose-shift response pattern. The win-stay portion was learned substantially more easily and appeared from the outset of training, suggesting that its occurrence did not depend upon discrimination of the local contingencies but rather only upon simple strengthening effects of individual reinforcements. Control by both types of local contingencies decreased with increases in the intertrial interval, although some control remained with intertrial intervals as long as 30 s. In Experiment 2, the local contingencies always favored win-shift and lose-shift response patterns but were asymmetrical for the two responses, causing the molar reinforcement rates for the two responses to differ. Some learning of the alternation pattern occurred with short intertrial intervals, although win-stay behavior occurred for some subjects. The local reinforcement contingencies were discriminated poorly with longer intertrial intervals. In the absence of control by the local contingencies, choice proportion was determined by the molar contingencies, as indicated by high exponent values for the generalized matching law with long intertrial intervals, and lower values with short intertrial intervals. The results show that when molar contingencies of reinforcement and local contingencies are in opposition, both may have independent roles. Control by molar contingencies cannot generally be explained by local contingencies.

show abstract

“…Another possibility is that the pigeons did not learn the stable patterns because frequency-dependent schedules are intrinsically challenging. To see this, consider the discrete-trials schedules commonly designed to test momentary maximizing (e.g., Hiraoka, 1984;Shimp, 1966;Silberberg & Williams, 1974;Williams, 1972Williams, , 1991. In most of these schedules, a salient event such as food or a response occurrence resets the reinforcement contingencies (Staddon, Hinson, & Kram, 1981), whereas in frequency-dependent schedules there is no such event.…”

Section: Choice Behaviormentioning

confidence: 99%

Polymorphic response patterns under frequency-dependent selection

Machado

1994

Animal Learning & Behavior

View full text Add to dashboard Cite

In a discrete-trials procedure, a frequency-dependent schedule shaped left-right choice proportion toward various equilibrium values between 0 and 1. At issue was (1) whether pigeons match when the overall reinforcement probabilities for two responses depend inversely on their recent frequency, and (2)how pigeons meet the schedule constraint in terms of local responding. That is, do they respond quasi-randomly (Bernoulli mode), or do they learn the stable pattern of the schedule (stable-pattern mode)? Molar choice behavior always tracked the equilibrium solution ofthe schedule, but the molecular response patterns varied substantially. Markov chains applied to the data revealed that responding was generally intermediate between the memoryless Bernoulli mode, and the perfect memory stable-pattern mode. The polymorphism of molecular patterns, despite molar regularities in behavior, suggests that (1) in order to engender the Bernoulli or stable-pattern modes, the reinforcement rule must strongly discourage competing response patterns (e.g., perseveration), and (2) under frequency-dependent schedules, molar matching is apparently not the outcome of momentary maximizing.Laboratory studies have shownthat frequency-dependent selection, the differential reinforcement of infrequent responses, sometimes engenders high degrees of behavioral variability. For example, when Page and Neuringer (1985) rewarded pigeons for generating sequences of eight left or right pecks that differed from the sequences generated on the last 50 trials, they obtained highly variable, random-like behavior. That is, the birds not only pecked each key equally often, but also showed few or no sequential dependencies between consecutive responses (see also Hest, Haaren, & Van De Poll, 1989;Morgan & Neuringer, 1990;Machado, 1989Machado, , 1992Morris, 1987Morris, , 1989Neuringer, 1991Neuringer, , 1992. However, frequencydependent selection may also engender stereotypical behavior. In a previous study (Machado, 1992), a frequencydependent schedule differentially reinforced the momentarily least likely of two responses-the more a pigeon preferred the right key, the less food it received from that key and the more food it received from the left key. Similar contingencies operated when preference shifted to the left; the probability of food for left (L) and right (R) pecks was equal only at indifference. Although the pigeons pecked each key on 50% of the trials, they did so by alternating rather than responding randomly.The preceding results illustrate two points. First, pigeons can track the overall probabilities of reinforcement for each of two responses when these probabilities I thank John Staddon and the other L.A.B. group members for their comments on earlier versions of the manuscript. The research was supported by grants to Duke University from NSF and NIMH (John Staddon, principal investigator). Correspondence concerning this article may be sent to A. Machado,

show abstract

Discrete-trial probability learning in rats: Effects of local contingencies of reinforcement

Cited by 7 publications

References 17 publications

Measuring reinforcement learning and motivation constructs in experimental animals: Relevance to the negative symptoms of schizophrenia

Measuring reinforcement learning and motivation constructs in experimental animals: Relevance to the negative symptoms of schizophrenia

Choice as a Function of Local Versus Molar Reinforcement Contingencies

Polymorphic response patterns under frequency-dependent selection

Contact Info

Product

Resources

About