Some factors in probability matching.

Rubinstein, Irvin

doi:10.1037/h0038962

Cited by 27 publications

(16 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We agree with Rubinstein (1959) that the reason that subjects in group 1 do not all maximize is that the conclusion that the reinforcing sequence is random is a very difficult one to arrive at. This is probably because problems with this solution do not usually appear in the subject's environment.…”

Section: Methods Of Comparisonsupporting

confidence: 75%

“…Studies using large nurriliers of trials report consistent overshooting of the PM value (Edwards, 1961;Wainer, 1967). It is also felt (RUbinstein, 1959) that the belief that the reward sequence is genuinely random is not readily accepted by the subject since problems with this solution do not usually appear in the subject's environment.…”

Section: Criticisms Of Probability Matchingmentioning

confidence: 99%

See 1 more Smart Citation

A Principal Components Analysis of Models and Men

Wainer¹

1968

ETS Research Bulletin Series

View full text Add to dashboard Cite

One consequence of stimulus sampling theory is the prediction of probability matching as the asymptotic response probability. That is, subjects tend to choose each alternative with the frequency of its reinforcement. This prediction has been tested in many experiments and the boundaries within which it holds have been delineated. However, the experiments which find probability matching have been criticized for doing one or more of several inappropriate things. It should be noted that probability matching is not a “rational” solution, in the sense that the behavior which maximizes gain would be always to choose the more frequently reinforced response. Some investigators have found it possible to devise experimental situations in which the subjects do come closer to maximizing behavior. This study attempts to differentiate the conditions which lead to “matching” behavior from those which lead to “maximizing” and to develop a mathematical model for this behavior. The experimental results indicate that the prediction of probability matching does not obtain. Therefore, it would seem to be probable that the stimulus sampling theory which predicts this behavior also does not apply. One of the axioms of stimulus sampling theory has been generalized. An effect of this revision is to allow for the prediction of nonprobability matching behavior. This model has been fitted to the data previously gathered, and described the data far better than any previous stimulus sampling model. It is concluded that probability matching is not a general learning law as Estes proposed in 1957, but, rather, that it is a special case of a more general phenomenon, just as previous stimulus sampling models are special cases of the model developed here. The major analytic tool in this study was a principal components analysis of the learning data. The inter‐factor plot of factor scores was used to compare the relative fit of the various forms of stimulus sampling models studied.

show abstract

Section: Methods Of Comparisonsupporting

confidence: 75%

Section: Criticisms Of Probability Matchingmentioning

confidence: 99%

A Principal Components Analysis of Models and Men

Wainer¹

1968

ETS Research Bulletin Series

View full text Add to dashboard Cite

show abstract

“…Because this is an intermediate between MA and MS we only consider it when computing the , where we distinguish between those. The second possibility would be to use PM for the action selection step, which was found in various studies to be a strategy employed by human observers in certain tasks [78], [79]. This is actually implicitly assumed in our model by using the softmax function to pick the action, thereby we do not include this option in our analysis.…”

Section: Methodsmentioning

confidence: 99%

Bayesian Cue Integration as a Developmental Outcome of Reward Mediated Learning

et al. 2011

View full text Add to dashboard Cite

Average human behavior in cue combination tasks is well predicted by Bayesian inference models. As this capability is acquired over developmental timescales, the question arises, how it is learned. Here we investigated whether reward dependent learning, that is well established at the computational, behavioral, and neuronal levels, could contribute to this development. It is shown that a model free reinforcement learning algorithm can indeed learn to do cue integration, i.e. weight uncertain cues according to their respective reliabilities and even do so if reliabilities are changing. We also consider the case of causal inference where multimodal signals can originate from one or multiple separate objects and should not always be integrated. In this case, the learner is shown to develop a behavior that is closest to Bayesian model averaging. We conclude that reward mediated learning could be a driving force for the development of cue integration and causal inference.

show abstract

“…This is an exceptionally consistent effect known as "probability matching." It has been replicated in dozens of laboratories, under myriad task conditions, and is extremely robust, persisting for thousands of trials (1)(2)(3)(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14)(15). Most theories treat this behavior as a fundamental failure of rational decision making.…”

mentioning

confidence: 99%

Alterations in choice behavior by manipulations of world model

Green

Benson²,

Kersten

et al. 2010

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

How to compute initially unknown reward values makes up one of the key problems in reinforcement learning theory, with two basic approaches being used. Model-free algorithms rely on the accumulation of substantial amounts of experience to compute the value of actions, whereas in model-based learning, the agent seeks to learn the generative process for outcomes from which the value of actions can be predicted. Here we show that (i) "probability matching"-a consistent example of suboptimal choice behavior seen in humans -occurs in an optimal Bayesian model-based learner using a max decision rule that is initialized with ecologically plausible, but incorrect beliefs about the generative process for outcomes and (ii) human behavior can be strongly and predictably altered by the presence of cues suggestive of various generative processes, despite statistically identical outcome generation. These results suggest human decision making is rational and model based and not consistent with model-free learning.decision making | probability matching | reinforcement learning G iven a limited set of data about the world, what is the best thing to do? This question lies at the heart of all decision making, from simple everyday errands to elaborate and complex scientific experiments. If the reward amount for each possible action is known in advance, it is a straightforward process to make choices that maximize reward. In the real world, however, reward values are nearly always initially unknown and computing them is not trivial. Thus, understanding how to learn and compute reward is one of the key problems in reinforcement learning theory. Computing the optimal policy (i.e., determining the "best thing to do") requires acquiring one of two types of knowledge. In model-free learning, an agent must accumulate a substantial amount of experience regarding the consequences of taking various actions in various states, from which the average value of the states can be learned. In model-based learning, an agent must acquire a "world model," which constitutes beliefs about how the world generates outcomes in response to actions. Although both model-free and model-based reinforcement-learning algorithms have been the subject of much study in computer science and machine learning, model-free algorithms have been primarily used as models of human choice behavior.Whereas it is clear that our survival depends on the ability to make appropriate decisions from incomplete and ambiguous information, numerous studies in economics, psychology, and neuroscience have consistently found highly suboptimal behavior in seemingly simple decision tasks. Why is this? Consider the sequential binary decision task, which involves a choice between two options, one with a higher probability of success than the other (e.g., 70% vs. 30% of trials). The optimal strategy for this task is to determine which option has a higher probability of success and then choose only that option. Humans, however, tend to sample the alternatives in proportion to the options' respec...

show abstract

Some factors in probability matching.

Cited by 27 publications

References 6 publications

A Principal Components Analysis of Models and Men

A Principal Components Analysis of Models and Men

Bayesian Cue Integration as a Developmental Outcome of Reward Mediated Learning

Alterations in choice behavior by manipulations of world model

Contact Info

Product

Resources

About