Optimal indolence: a normative microscopic approach to work and leisure

Niyogi, Ritwik K.; Breton, Yannick-André; Solomon, Rebecca Brana; Conover, Kent; Shizgal, Peter; Dayan, Peter

doi:10.1098/rsif.2013.0969

Cited by 19 publications

(31 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These results tie decision-making in this class of tasks to a diverse set of behavioral phenomena and neural mechanisms that have been previously associated with average reward rates. Notably, the average reward rate has arisen in the analysis of a number of other seemingly disparate behaviors, including behavioral vigor, risk sensitivity, labor-leisure tradeoffs, self-control, and time discounting (Niv et al, 2006; Niv et al, 2007; Cools et al, 2011; Kacelnik, 1997; Niyogi et al, 2014; Keramati et al, 2011; Kurzban et al, 2012; Gallistel and Gibbon, 2000; Daw and Touretzky, 2002; Guitart-Masip et al, 2011). The notion of the average reward rate as the opportunity cost of time links these seemingly disparate domains: it indicates, for instance, the potential reward foregone by behaving less vigorously, by waiting for a delayed reward in a discounting task, or, in the case of foraging, by spending more time with the current option rather than searching for a better one (Niv et al, 2006; Niv et al, 2007).…”

Section: Discussionmentioning

confidence: 99%

Learning the opportunity cost of time in a patch-foraging task

Constantino

Daw

2015

Cogn Affect Behav Neurosci

190

293

View full text Add to dashboard Cite

Although most decision research concerns choice between simultaneously presented options, in many situations options are encountered serially and the decision is whether to exploit an option or search for a better one. Such problems have a rich history in animal foraging but we know little about the psychological processes involved. In particular, it is unknown whether learning in these problems is supported by the well studied neurocomputational mechanisms involved in more conventional tasks. We investigated how humans learn in a foraging task, which requires deciding whether to harvest a depleting resource or switch to a replenished one. The optimal choice (given by the Marginal Value Theorem; MVT) requires comparing the immediate return from harvesting to the opportunity cost of time, which is given by the long-run average reward. In two experiments, we varied opportunity cost across blocks. Subjects adjusted their behavior to blockwise changes in environmental characteristics. We examined how subjects learned their choice strategies by comparing choice adjustments to a learning rule suggested by the MVT (where the opportunity cost threshold is estimated as an average over previous rewards) and to the predominant incremental learning theory in neuroscience, temporal-difference learning (TD). Trial-by-trial decisions were better explained by the MVT threshold learning rule. These findings expand on the foraging literature, which has focused on steady-state behavior, by elucidating a computational mechanism for learning in switching tasks that is distinct from those used in traditional tasks, and suggest connections to research on average reward rates in other domains of neuroscience.

show abstract

Section: Discussionmentioning

confidence: 99%

Learning the opportunity cost of time in a patch-foraging task

Constantino

Daw

2015

Cogn Affect Behav Neurosci

190

293

View full text Add to dashboard Cite

show abstract

“…Thus, the agent is optimizing the intertemporal tradeoff between acting (providing a more immediate opportunity for reward) and replaying (providing an opportunity for greater but later reward). This insight may help to rationalize the labor and leisure tradeoff that has been described for cognitive control (Kool & Botvinick, 2014;Niyogi, Breton, et al, 2014;Inzlicht, Schmeichel, & Macrae, 2014;Dora, van Hooff, Geurts, Kompier, & Bijleveld, 2019). Results.…”

Section: Hippocampal Replaymentioning

confidence: 95%

The Temporal Dynamics of Opportunity Costs: A Normative Account of Cognitive Fatigue and Boredom

Agrawal

Mattar

Cohen

et al. 2020

Preprint

View full text Add to dashboard Cite

Cognitive fatigue and boredom are two phenomenological states widely associated with limitations in cognitive control. In this paper, we present a rational analysis of the temporal structure of controlled behavior, which provides a new framework for providing a formal account of these phenomena. We suggest that in controlling behavior, the brain faces competing behavioral and computational imperatives, and must balance them by tracking their opportunity costs over time. We use this analysis to flesh out previous suggestions that feelings associated with subjective effort, like cognitive fatigue and boredom, are the phenomenological counterparts of these opportunity cost measures, rather then reflecting the depletion of resources as has often been assumed. Specifically, we propose that both fatigue and boredom reflect the competing value of particular options that require foregoing immediate reward but can improve future performance: Fatigue reflects the value of offline computation (internal to the organism) to improve future decisions, while boredom signals the value of exploratory actions (external in the world) to gather information. We demonstrate that these accounts provide a mechanistically explicit and parsimonious account for a wide array of findings related to cognitive control, integrating and reimagining them under a single, formally rigorous framework.

show abstract

“…Relatively little is known about the neural basis of serial stay-switch decisions, although researchers have recently argued that they engage distinct cortical mechanisms from extensively studied simultaneous choice (Hayden et al, 2011; Kolling et al, 2012). A separate, related literature has centered on decisions about effort: How vigorously to work (Robbins, 1976; Niv et al, 2007; Guitart-Masip et al, 2011; Niyogi et al, 2014). Such decisions should in theory be governed by parallel opportunity cost considerations, since sloth becomes more costly as earnings potential increases.…”

Section: Introductionmentioning

confidence: 99%