Reward and Feedback in the Control over Dynamic Events

Abstract

On what basis do we learn to make effective decisions when faced with intermittent feedback from actions taken in a dynamic decision-making environment? In the present study we hypothesize that reward information (financial, social) may provide useful signals that can guide decision-making in these situations. To examine this, we present three experiments in which people make decisions directly towards controlling a dynamically uncertain output. We manipulate the framing of incentives (gains and losses) and the form of the incentives (financial, social), and measure their impact on decision-making performance (both in frequent and intermittent output feedback conditions). Overall, performance suffered under intermittent output feedback. Relative to social rewards, financial rewards generally improved control performance, and a gains framing (financial, social) leading to better performance than a losses framing (financial, social). To understand how rewards affect behavior in our tasks, we present a reinforcement learning model to capture the learning and performance profiles in each of our experiments. This study shows that information regarding incentives impacts the levels of exploration, the optimality of decisions-making, and the variability in the strategies people develop to control a dynamic output experienced frequently and intermittently.

Justice at any cost? The impact of cost–benefit salience on criminal punishment judgments

“…Research suggests that experts, including judges, are not immune to heuristic reasoning. Experts have demonstrated susceptibility to many of the same heuristic processes that shape lay reasoning, such as anchoring, base rate neglect, and opportunity cost neglect (Bennett, 2014;Northcraft & Neale, 1987;Vera-Muñoz, 1998;West et al, 2012;Wong, Aharoni, Aliev, & DuBois, 2015). Thus, it would not be especially surprising if punishment recommendations by professional judges are also affected by cues that increase the saliency of the costs of incarceration.…”

“…By this view, it seems to follow that people who excel in executive functions such as delayed gratification, counterfactual reasoning (i.e., the tendency to entertain multiple possible perspectives or outcomes), or emotion regulation (i.e., the tendency to control one's emotional reactions to affective stimuli) should exhibit less susceptibility to such biases because they should be better able to attend to and incorporate factors relevant to their decision regardless of its salience. Yet support for this hypothesis is mixed, as other research suggests that individuals with high cognitive ability or subject-matter expertise are no less susceptible to many cognitive biases and may even be more susceptible in some cases (Bennett, 2014;Vera-Muñoz, 1998;West, Meserve, & Stanovich, 2012). Further research is needed to understand the extent to which sentencing decisions are affected by executive function abilities.…”

“…Second, just as decision makers can overweight extraneous information, they can also underweight potentially relevant information. For example, research in business contexts has shown that, when reasoning about the costs and benefits of a costly decision, people tend to discount the decision's expected costs, including, for instance, the opportunities that were foregone in order to pay for the one that is selected (Becker, Ronen, & Sorter, 1974;Frederick et al, 2009;Hoskin, 1983;Jones, Frisch, Yurak, & Kim, 1998;Neumann & Friedman, 1978;Northcraft & Neale, 1986;Pontari, Stanaland, & Smythe, 2009;Vera-Muñoz, 1998). To our knowledge, such cost discounting has never been investigated in the context of criminal sentencing, where money that is spent on a prison sentence can no longer be used to invest in other government-funded programs such as policing, job training, healthcare, and other reentry services for former inmates.…”

A test of synergy in dynamic system control tasks.

“…In addition, variable relations in more recent dynamic system control tasks are frequently linear and transparent, and they often lack time-delays and side effects. For example, Osman and colleagues used a dynamic system in which participants controlled three input variables in order to optimize one outcome variable (Osman, Glass, & Hola, 2015; Osman, Glass, Hola, & Stollewerk, 2017). In this system, the outcome is a linear combination of the prior value of the outcome variable, two of the three input variables, and added noise, which represented the system’s own dynamics.…”

“…However, this conclusion may not be generally true. Some of the more recent studies paint a more optimistic picture of system control performance, suggesting that individuals can manage less complex dynamic systems quite well, even with rather impoverished information (e.g., Osman et al, 2015Osman et al, , 2017.…”