On Adaptation, Maximization, and Reinforcement Learning Among Cognitive Strategies.

Abstract

Analysis of binary choice behavior in iterated tasks with immediate feedback reveals robust deviations from maximization that can be described as indications of 3 effects: (a) a payoff variability effect, in which high payoff variability seems to move choice behavior toward random choice; (b) underweighting of rare events, in which alternatives that yield the best payoffs most of the time are attractive even when they are associated with a lower expected return; and (c) loss aversion, in which alternatives that minimize the probability of losses can be more attractive than those that maximize expected payoffs. The results are closer to probability matching than to maximization. Best approximation is provided with a model of reinforcement learning among cognitive strategies (RELACS). This model captures the 3 deviations, the learning curves, and the effect of information on uncertainty avoidance. It outperforms other models in fitting the data and in predicting behavior in other experiments.

How choice ecology influences search in decisions from experience

“…The higher the outcome variance of risky options (with otherwise equal expected values), that is, the riskier it is, the less likely people are to choose the options, and the lower their willingness to pay for them (Sharpe, 1964). This ''payoff-variability effect'' is also observed when people experience variance while choosing repeatedly from a set of options (Busemeyer & Townsend, 1993;Erev & Barron, 2005;Myers & Sadler, 1960).…”

Testing adaptive toolbox models: A Bayesian hierarchical approach.

“…In the social domains, researchers have argued that people rely on different strategies for social interactions (Erev & Roth, 2001;Fiske, 1992;Milinski & Wedekind, 1998), mating choices (Buss & Schmitt, 1993), and predicting other people's behavior (Costa-Gomes & Crawford, 2006). Furthermore, researchers have argued that people use different strategies for categorization (Busemeyer & Myung, 1992;Patalano, Smith, Jonides, & Koeppe, 2001;Sewell & Lewandowsky, 2011), resource allocation (Ball, Langholtz, Auble, & Sopchak, 1998), estimation and frequency judgments (Brown, 1995;Brown, Cui, & Gordon, 2002;von Helversen & Rieskamp, 2008), skill acquisition (Anderson & Lebiere, 1998;Lovett, 1988), function learning (Lewandowsky, Kalish, & Ngang, 2002), and learning processes (Erev & Barron, 2005;Gigerenzer & Gaissmaier, 2011). The idea that processes such as information search or choice are guided by qualitatively different strategies also figures prominently in research on judgment and decision making.…”

Decision making and learning while taking sequential risks.

“…It can, for example, create an aversion toward risky alternatives in the gain domain and an attraction toward risky alternatives in the loss domain-a pattern typically attributed to how DMs evaluate outcomes (Denrell, 2007;March, 1996). The learning process can even produce the opposite pattern (Erev & Barron, 2005;Hertwig, Barron, Weber, & Erev, 2004;Weber, Shafir, & Blais, 2004).Applying theories of decision making to the Balloon Analogue Risk Task (BART; Lejuez et al, 2002) or to the Iowa Gambling Task (Bechara, Damasio, Damasio, & Anderson, 1994) also exposes the necessity of learning. Clinicians use these laboratorybased gambling tasks to study and identify people with specific clinical or neurological deficits.…”