Optimal behavior in free-operant experiments.

Shimp, Charles P.

doi:10.1037/h0027311

Cited by 222 publications

(148 citation statements)

References 50 publications

(48 reference statements)

Supporting

Mentioning

141

Contrasting

Order By: Relevance

“…In the literature on operant behavior, such a view is referred to as "molar", to distinguish it from "molecular" analysis, which is concerned with moment-tomoment decisions. Shimp (1969) claimed that momentary maximization of the probability of reward on VI schedules would result in matching. We believe that Shimp's use of the term 'optimal' in this context has led to an unwarranted opposition between molar (i.e., global) and molecular (i.e., immediate or momentary) maximization.…”

Section: Discrete Trial Conc VI Vi Immediate and Overall Maximizationmentioning

confidence: 99%

“…Immediate Maximization Shimp (1969) suggested that organisms should behave optimally in the sense of always making the choice that momentarily has the greatest expected value. This principle is known as immediate or molecular maximization.…”

Section: Discrete Trial Conc VI Vi Immediate and Overall Maximizationmentioning

confidence: 99%

“…This principle is known as immediate or molecular maximization. In the case of the discrete VI problem we are considering here, immediate maximization predicts a fixed sequence of responses which involves approximate matching to the programmed reinforcement rates (Shimp, 1969;Staddon, 1980 Table 9 n (the optimal value of n,, first number) and n(im) (the value of n, predicted by immediate maximization, second number) for various values of q, and q2.…”

Section: Discrete Trial Conc VI Vi Immediate and Overall Maximizationmentioning

confidence: 99%

See 2 more Smart Citations

How to Maximize Reward Rate on Two Variable‐interval Paradigms

Houston

McNamara

1981

J Exper Analysis Behavior

173

121

View full text Add to dashboard Cite

Without assuming any constraints on behavior, we derive the policy that maximizes overall reward rate on two variable-interval paradigms. The first paradigm is concurrent variable time-variable time with changeover delay. It is shown that for nearly all parameter values, a switch to the schedule with the longer interval should be followed immediately by a switch back to the schedule with the shorter interval. The matching law does not hold at the optimum and does not uniquely specify the obtained reward rate. The second paradigm is discrete trial concurrent variable interval-variable interval. For given schedule parameters, the optimal policy involves a cycle of a fixed number of choices of the schedule with the shorter interval followed by one choice of the schedule with the longer interval. Molecular maximization sometimes results in optimal behavior.

show abstract

Section: Discrete Trial Conc VI Vi Immediate and Overall Maximizationmentioning

confidence: 99%

Section: Discrete Trial Conc VI Vi Immediate and Overall Maximizationmentioning

confidence: 99%

Section: Discrete Trial Conc VI Vi Immediate and Overall Maximizationmentioning

confidence: 99%

See 1 more Smart Citation

How to Maximize Reward Rate on Two Variable‐interval Paradigms

Houston

McNamara

1981

J Exper Analysis Behavior

173

121

View full text Add to dashboard Cite

show abstract

“…Several "molecular" or "quasi-molecular" accounts of matching, which posit maximizing payoffs at a local or molecular level, have been proposed. For example, meloriation (Herrnstein & Vaughan, 1980;Vaughan, 1985) and "momentary maximizing" (Shimp, 1969) argue that subjects choose the better of two local rates or probabilities of reinforcement, and this results in overall matching. Rather than explaining it, other, "molar" matching accounts simply assume matching (e.g., Baum & Rachlin, 1969; Gallistel, 1990;Mark & Gallistel, 1994).…”

mentioning

confidence: 99%

Dynamics of time matching: Arousal makes better seem worse

Gibbon

1995

Psychon Bull Rev

115

View full text Add to dashboard Cite

Matching of time allocation across alternatives in proportion to relative reinforcement rates is a ubiquitous finding in the animal-learning literature on choice. The dynamics ofthe underlying mechanism, however, remain poorly understood. A recent finding by Belke (1992) profoundly challenges scalar expectancy theory (SET; Gibbon et al., 1988) and other accounts of matching in concurrent variable interval (VI) schedules. He studied concurrent probe tests of stimuli associated with equal VIs but trained in alternative concurrent pairs. In training, one was preferred and the other not. Unreinforced probes revealed a strong preference for the alternative preferred in training. An experiment is reported replicating this result and showing that it is not due to generalization of preference levels from training. When the probe is between the two preferred training stimuli, the richer schedule is unpreferred. A SET account of these results is presented which implicates two processes in time allocation: (1) the choice between alternatives based on memory for delays to reinforcement, and (2) the times at which such choices are made. The former process is sensitive to reinforcement scheduling; the latter is sensitive to arousal levels induced by overall reinforcement rates in training.A now venerable tradition in operant schedule research shows that animals choosing between concurrently available variable schedules of reward allocate time spent at each alternative in proportion to their rates ofpayoff(the matching law; Herrnstein, 1970; see Williams, 1988, for a review). The matching law is ubiquitous throughout the animal kingdom. A form of matching is seen at the level of groups of animals distributing individuals across resource sites where it is known as the "ideal free distribution" (see Gallistel, 1990, for a review). At the level of individuals, where it has been studied extensively in the animal-learning literature, accounts of matching have engendered an enduring controversy about its source. Matching of behavior proportionate to payoff implies more choices for the less profitable resource than strictly necessary to maximize overall payoff rates. Several "molecular" or "quasi-molecular" accounts of matching, which posit maximizing payoffs at a local or molecular level, have been proposed. For example, meloriation (Herrnstein & Vaughan, 1980;Vaughan, 1985) and "momentary maximizing" (Shimp, 1969) argue that subjects choose the better of two local rates or probabilities of reinforcement, and this results in overall matching. Rather than explaining it, other, "molar" matching accounts simply assume matching (e.g., Baum & Rachlin, 1969; Gallistel, 1990;Mark & Gallistel, 1994).The author is grateful for extensive discussion of these ideas with C. R. Gallistel, who lent formative insights into an understanding of switching rates. Thanks are due S. Fairhurst for data collection and analysis. The work was supported by NIMH Grant MH41649-09. Address correspondence to John Gibbon, Biopsychology Unit 50, 722 W. I68th St.,...

show abstract

“…Skinner's "kinds of evidence" for strength -including whether a response is emitted at all, its energy level, its speed, the amount of repetition, and its overall frequency -suggest that strength is a property underlying observed behavior. Constructions such as "evidence of strength," "sign of strength," and "indicator of strength" occur at least 15 times in Verbal Behavior, making strength come across as a kind of "state variable" which is not directly observable (Shull, 1993) Shimp, 1969), stimulus generalization (e.g., Migler, 1964), and so-called continuous repertoires (Wildemann & Holland, 1972).…”

mentioning

confidence: 99%

The concept of strength in the analysis of verbal behavior

Cherpas

1993

Analysis Verbal Behav

View full text Add to dashboard Cite

A textual analysis of the concept of strength is presented, based on patterns of its occurrence in the book Verbal Behavior (Skinner, 1957). A manual count was conducted of all instances of the word "strength," and closely related forms. Rates were then plotted and interpreted as revealing the kinds of situations where strength is most relevant. Strength appears to be most relevant (as measured by instances per page of text) whenever a detailed behavioral analysis involves 1) more than one source of strength for a response, 2) multiple responses or fragments being strengthened by one or more variables, 3) dynamic changes in behavior, or 4) behavior which is not currently or readily observed. Further research is needed to evaluate how textual analysis of this sort contributes to a science of behavior.

show abstract

Optimal behavior in free-operant experiments.

Cited by 222 publications

References 50 publications

How to Maximize Reward Rate on Two Variable‐interval Paradigms

How to Maximize Reward Rate on Two Variable‐interval Paradigms

Dynamics of time matching: Arousal makes better seem worse

The concept of strength in the analysis of verbal behavior

Contact Info

Product

Resources

About