Small feedback‐based decisions and their limited correspondence to description‐based decisions

Abstract

The present paper explores situations in which the information available to decision makers is limited to feedback concerning the outcomes of their previous decisions. The results reveal that experience in these situations can lead to deviations from maximization in the opposite direction of the deviations observed when the decisions are made based on a description of the choice problem. Experience was found to lead to a reversed common ratio/certainty effect, more risk seeking in the gain than in the loss domain, and to an underweighting of small probabilities. Only one of the examined properties of description-based decisions, loss aversion, seems to emerge robustly in these 'feedback-based' decisions. These results are summarized with a simple model that illustrates that all the unique properties of feedback-based decisions can be a product of a tendency to rely on recent outcomes. Copyright # 2003 John Wiley & Sons, Ltd. key words Probability learning; feedback-based decisions; reinforcement learning; prospect theory; Allais paradox Many common activities involve 'small' decision problems. Driving, for example, requires repeated selection among routes, speeds, and various other options. Although little time and effort is typically invested in these and similar small decisions, they can be consequential. The estimated cost of traffic accidents in the USA is more than 100 billion dollars a year (see e.g. Blincoe, 1994 1 ), and many of the accidents are at least partially products of ex-post unwise decisions.The current paper focuses on an important subset of the small decision problems exemplified above that can be referred to as 'small feedback-based' decisions. These problems are defined by three main properties. First, they are repeated; decision makers face the same problem many times in similar situations. Second,

Seeing is believing: Trustworthiness as a dynamic belief

“…Because the model-predicted ratings update based on experience, they were also better able to predict the post-experiment probability ratings than the initial trustworthiness ratings. These findings support and extend previous research, which has found that experience overwhelms description in risky choice (Barron & Erev, 2003;Hertwig, Barron, Weber, & Erev, 2004;Jessup, Bishara, & Busemeyer, 2008). While this notion of dynamically updating beliefs is certainly not new, this study provides, to our knowledge, support for the first computational model of this effect in an iterative social exchange.…”

From anomalies to forecasts: Toward a descriptive model of decisions under risk, under ambiguity, and from experience.

“…An experiment that evaluates (and rejects) this "expected repetitions" hypothesis is presented in the section Control Conditions and Robustness Checks below. Figure 3 also shows that the emergence of underweighting of rare events in decisions with feedback is robust (Barron & Erev, 2003;Lejarraga & Gonzalez, 2011;and see Row 11 in Table 1). Experience reduces sensitivity to the rare event in all five problems.…”

Learning to cooperate without awareness in multiplayer minimal social situations

“…Apart from reinforcement learning, these approaches are inapplicable to the MSS because of the impoverished information that is available to MSS players. Simple reinforcement learning theories (e.g., Barron & Erev, 2003;Erev & Roth, 1998;Feltovich, 2000;Roth & Erev, 1995), on the other hand, are closely related to WSLS, and the most relevant forms of reinforcement learning will be revisited in Section 6.4.…”

“…Some of these models are capable of predicting the results of our experiments. Barron and Erev (2003) developed a value-assessment model that they showed to be successful at predicting repetitive individual decisions when information available to decision makers is limited to feedback about outcomes on previous trials. In the value-assessment model, the value of a decision j is initially the subjective expected value from random choice; then the adjusted value A j on trial t + 1 is A j (t + 1) = (1 -w t )A j (t) + (w t )v(x t ), where v(x t ) is the subjective value of the payoff x t weighted by 0 < w t < 1.…”