The smart potential behind probability matching

Gaissmaier, Wolfgang; Schooler, Lael J.

doi:10.1016/j.cognition.2008.09.007

Cited by 180 publications

(193 citation statements)

References 27 publications

(29 reference statements)

Supporting

Mentioning

169

Contrasting

Unclassified

Order By: Relevance

“…Pðx is goodÞ Pðx is goodÞ + Pðy is goodÞ : [2] This rule is known to be optimal when there is competition for resources (39,40) and when the estimated probabilities change in time (41)(42)(43)(44). Probability matching in Eq.…”

Section: Resultsmentioning

confidence: 99%

A common rule for decision making in animal collectives across species

Arganda

Pérez‐Escudero

Polavieja

2012

Proc. Natl. Acad. Sci. U.S.A.

169

210

View full text Add to dashboard Cite

A diversity of decision-making systems has been observed in animal collectives. In some species, choices depend on the differences of the numbers of animals that have chosen each of the available options, whereas in other species on the relative differences (a behavior known as Weber’s law), or follow more complex rules. We here show that this diversity of decision systems corresponds to a single rule of decision making in collectives. We first obtained a decision rule based on Bayesian estimation that uses the information provided by the behaviors of the other individuals to improve the estimation of the structure of the world. We then tested this rule in decision experiments using zebrafish ( Danio rerio ), and in existing rich datasets of argentine ants ( Linepithema humile ) and sticklebacks ( Gasterosteus aculeatus ), showing that a unified model across species can quantitatively explain the diversity of decision systems. Further, these results show that the different counting systems used by animals, including humans, can emerge from the common principle of using social information to make good decisions.

show abstract

Section: Resultsmentioning

confidence: 99%

A common rule for decision making in animal collectives across species

Arganda

Pérez‐Escudero

Polavieja

2012

Proc. Natl. Acad. Sci. U.S.A.

169

210

View full text Add to dashboard Cite

show abstract

“…Or perhaps they felt that maximizing was too simple a strategy to be the optimal solution to the decision task they faced. It has also been suggested, although it does not apply to the context of our experiments, that matching may arise from a (futile) search for patterns in the random outcome sequence (see, e.g., Gaissmaier & Schooler, 2008). All of these possibilities result from what might be considered forms of overthinking rather than underthinking.…”

Section: Methodsmentioning

confidence: 75%

Probability matching and strategy availability

Koehler

James

2010

Memory & Cognition

View full text Add to dashboard Cite

“…This is an exceptionally consistent effect known as "probability matching." It has been replicated in dozens of laboratories, under myriad task conditions, and is extremely robust, persisting for thousands of trials (1)(2)(3)(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14)(15). Most theories treat this behavior as a fundamental failure of rational decision making.…”

mentioning

confidence: 99%

Alterations in choice behavior by manipulations of world model

Green

Benson²,

Kersten

et al. 2010

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

How to compute initially unknown reward values makes up one of the key problems in reinforcement learning theory, with two basic approaches being used. Model-free algorithms rely on the accumulation of substantial amounts of experience to compute the value of actions, whereas in model-based learning, the agent seeks to learn the generative process for outcomes from which the value of actions can be predicted. Here we show that (i) "probability matching"-a consistent example of suboptimal choice behavior seen in humans -occurs in an optimal Bayesian model-based learner using a max decision rule that is initialized with ecologically plausible, but incorrect beliefs about the generative process for outcomes and (ii) human behavior can be strongly and predictably altered by the presence of cues suggestive of various generative processes, despite statistically identical outcome generation. These results suggest human decision making is rational and model based and not consistent with model-free learning.decision making | probability matching | reinforcement learning G iven a limited set of data about the world, what is the best thing to do? This question lies at the heart of all decision making, from simple everyday errands to elaborate and complex scientific experiments. If the reward amount for each possible action is known in advance, it is a straightforward process to make choices that maximize reward. In the real world, however, reward values are nearly always initially unknown and computing them is not trivial. Thus, understanding how to learn and compute reward is one of the key problems in reinforcement learning theory. Computing the optimal policy (i.e., determining the "best thing to do") requires acquiring one of two types of knowledge. In model-free learning, an agent must accumulate a substantial amount of experience regarding the consequences of taking various actions in various states, from which the average value of the states can be learned. In model-based learning, an agent must acquire a "world model," which constitutes beliefs about how the world generates outcomes in response to actions. Although both model-free and model-based reinforcement-learning algorithms have been the subject of much study in computer science and machine learning, model-free algorithms have been primarily used as models of human choice behavior.Whereas it is clear that our survival depends on the ability to make appropriate decisions from incomplete and ambiguous information, numerous studies in economics, psychology, and neuroscience have consistently found highly suboptimal behavior in seemingly simple decision tasks. Why is this? Consider the sequential binary decision task, which involves a choice between two options, one with a higher probability of success than the other (e.g., 70% vs. 30% of trials). The optimal strategy for this task is to determine which option has a higher probability of success and then choose only that option. Humans, however, tend to sample the alternatives in proportion to the options' respec...

show abstract

The smart potential behind probability matching

Cited by 180 publications

References 27 publications

A common rule for decision making in animal collectives across species

A common rule for decision making in animal collectives across species

Probability matching and strategy availability

Alterations in choice behavior by manipulations of world model

Contact Info

Product

Resources

About