Learning the opportunity cost of time in a patch-foraging task

Abstract

Although most decision research concerns choice between simultaneously presented options, in many situations options are encountered serially and the decision is whether to exploit an option or search for a better one. Such problems have a rich history in animal foraging but we know little about the psychological processes involved. In particular, it is unknown whether learning in these problems is supported by the well studied neurocomputational mechanisms involved in more conventional tasks. We investigated how humans learn in a foraging task, which requires deciding whether to harvest a depleting resource or switch to a replenished one. The optimal choice (given by the Marginal Value Theorem; MVT) requires comparing the immediate return from harvesting to the opportunity cost of time, which is given by the long-run average reward. In two experiments, we varied opportunity cost across blocks. Subjects adjusted their behavior to blockwise changes in environmental characteristics. We examined how subjects learned their choice strategies by comparing choice adjustments to a learning rule suggested by the MVT (where the opportunity cost threshold is estimated as an average over previous rewards) and to the predominant incremental learning theory in neuroscience, temporal-difference learning (TD). Trial-by-trial decisions were better explained by the MVT threshold learning rule. These findings expand on the foraging literature, which has focused on steady-state behavior, by elucidating a computational mechanism for learning in switching tasks that is distinct from those used in traditional tasks, and suggest connections to research on average reward rates in other domains of neuroscience.

Sympathetic and parasympathetic involvement in time constrained sequential foraging

“…In each model, was initialized at the beginning of the experiment to the arithmetic average reward rate across the experiment, but subsequently carried over between environments (Constantino and Daw, 2015;Garrett and Daw, 2019). For each participant, we estimated the free parameters of the To formally test for differences in learning rates ( p , M ) we estimated the covariance matrix Σ x over the group level parameters using the Hessian of the model likelihood (Oakes, 1999) and then used a contrast c z Σ x c to compute the standard error on the difference p -M .…”

“…Recent work in humans further resolves the computational challenge of learning dynamic environmental richness, by demonstrating that sequential choice behavior is best captured by an MVT inspired learning model. Specifically, both decisions to leave a patch and explore the environment in patch foraging (Constantino and Daw, 2015;Lenow et al, 2017) and capture behavior in prey selection (Garrett and Daw, 2019) adhere to the MVT predicted optimality policy that compares yields against fluctuating environmental richness which is learned via a standard delta rule. This later work further demonstrated that beliefs about environmental richness update with asymmetric bias, whereby improvements are learned at a higher rate than deteriorations; the 'naïve perseverance of optimism' (Garrett and Daw, 2019).…”

“…In addition, contexts like the boom and downturn examples above rarely remain in steady-state, further requiring perturbation-driven behaviour adjustments. Recent work in humans resolves both issues, by demonstrating that sequential choice behaviour is best captured by an MVT inspired learning model (Constantino & Daw, 2015;Garrett & Daw, 2019;Lenow, Constantino, Daw & Phelps, 2017). Specifically, the decision to depart one patch of potential resources to harvest in another during a dynamic patch foraging task is predicated on a leave threshold, defined as the harvest sacrificed when leaving.…”

“…Specifically, the decision to depart one patch of potential resources to harvest in another during a dynamic patch foraging task is predicated on a leave threshold, defined as the harvest sacrificed when leaving. The threshold adheres to an optimality policy that compares harvests against fluctuating environmental richness (i.e., the average starting yield of iteratively depleting patches) which are learned via a standard delta rule (Constantino & Daw, 2015;Lenow et al, 2017). Garrett and Daw (2019) later extended the application of MVT learning models to a prey selection task, and demonstrated that sequential capture behaviour also adheres to trialwise appraisal of prey quality relative to learned beliefs about fluctuating environmental richness (i.e., the concentration of hi or low quality prey).…”

Actitud hacia la marca ante un evento inesperado: Rol del pensamiento holístico y analítico

“…La decisión implica el optar por una opción, o conjunto de ellas, pero en tal elección lo que se evalúa en realidad es la renuncia o perdida por preferir un camino o ruta por sobre otras posibilidades (Spiller, 2011). Ya sea que se trate de una empresa o un individuo, distintas variables se incluyen en la noción del costo de oportunidad, y cada sujeto posee sus propios imaginarios acerca de tales variables, pero en términos generales, dos aspectos son tenidos en cuenta por parte del decisor cuando su elección se realiza en función alcosto de oportunidad: tiempo y dinero (Constantino y Daw, 2015;Frederick, Novemsky, Wang, Dhar y Nolwis, 2009;Mogilner y Aaker, 2009;Monga y Saini, 2009;Okada y Hoch, 2004;Payne, Bettman y Luce, 1996;Sanabria, Thrailkill y Killeen, 2009).…”