Control over patch encounters changes foraging behaviour

Hall-McMaster, Sam; Dayan, Peter

doi:10.1101/2021.01.19.426950

Cited by 4 publications

(6 citation statements)

References 55 publications

(110 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is thus highly doubtful that we are studying choices that are relevant for real world behavior. This is also true for putatively more ecologically valid foraging choices (Constantino & Daw, 2015;Kolling et al, 2012;Shenhav et al, 2016;Yoon, Geary, Ahmed, & Shadmehr, 2018), as demonstrated with the learnable version we reviewed earlier (Hall-McMaster et al, 2021). These studies reveal the additional computations and cognitive resources supporting decision-making that we have so far overlooked, but that are ripe for study.…”

Section: Decisions and Control Over Future Research Directionssupporting

confidence: 57%

“…One example is a recent study that modified the standard foraging paradigm, where people need to decide when to abandon a depleting patch for an unknown new one (Kolling et al, 2012;Shenhav et al, 2016; so that people could revisit previous patches and decide where to forage next. Hall-McMaster, Dayan, and Schuck (2021) showed that people learn and represent reward rates of different patches and use this information to choose patches that maximize reward. Importantly, they also leverage information about the rewards at other patches to adaptively adjust decisions about when to leave the current patch.…”

Section: Monitoring Over Multiple Levels Of a Response Hierarchymentioning

confidence: 99%

See 1 more Smart Citation

Filling the gaps: Cognitive control as a critical lens for understanding mechanisms of value-based decision-making

Frömer¹,

Shenhav²

2021

Preprint

View full text Add to dashboard Cite

Research into value-based decision making has made tremendous progress in identifying behavioral and neural correlates of choice value. However, these correlates have been primarily viewed through a field-specific lens, focusing on how they contribute to the evaluation and selection between options to arrive at a choice. Here, we reveal blind-spots resulting from this limited perspective, and how they can be filled in through taking the perspective of cognitive control. We highlight three particular insights that this perspective offers: (1) a view towards the goal-relevance of one’s options and their features; (2) a view of decision-making correlates as a proxy for monitoring to determine control adjustments; (3) a view of those correlates as a proxy for monitoring that extends temporally and hierarchically beyond the immediate choice task. We show how adopting these complementary perspectives offers new insight into the determinants of both decisions and control; alternative interpretations for common findings in the neuroeconomic literature; and fruitful directions for future research.

show abstract

Section: Decisions and Control Over Future Research Directionssupporting

confidence: 57%

Section: Monitoring Over Multiple Levels Of a Response Hierarchymentioning

confidence: 99%

Filling the gaps: Cognitive control as a critical lens for understanding mechanisms of value-based decision-making

Frömer¹,

Shenhav²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…However, theoretical work [ 52 ] shows that for patch foraging agents the optimal strategy should take into account the replenishing rates of unvisited patches. Supported by experimental work [ 53 ] manipulation of replenishing rates on different unvisited patches results in more frequent visits to fast replenishing sites as well as early leaving times in human subjects. While these studies did not examine the role of the choice history effects in maximizing the reward harvesting efficiency, these findings are consistent with our data that behavior of animal (choice history effects) is sensitive to growth rate of probabilities on unchosen options.…”

Section: Discussionmentioning

confidence: 99%

Choice history effects in mice and humans improve reward harvesting efficiency

et al. 2021

View full text Add to dashboard Cite

Choice history effects describe how future choices depend on the history of past choices. In experimental tasks this is typically framed as a bias because it often diminishes the experienced reward rates. However, in natural habitats, choices made in the past constrain choices that can be made in the future. For foraging animals, the probability of earning a reward in a given patch depends on the degree to which the animals have exploited the patch in the past. One problem with many experimental tasks that show choice history effects is that such tasks artificially decouple choice history from its consequences on reward availability over time. To circumvent this, we use a variable interval (VI) reward schedule that reinstates a more natural contingency between past choices and future reward availability. By examining the behavior of optimal agents in the VI task we discover that choice history effects observed in animals serve to maximize reward harvesting efficiency. We further distil the function of choice history effects by manipulating first- and second-order statistics of the environment. We find that choice history effects primarily reflect the growth rate of the reward probability of the unchosen option, whereas reward history effects primarily reflect environmental volatility. Based on observed choice history effects in animals, we develop a reinforcement learning model that explicitly incorporates choice history over multiple time scales into the decision process, and we assess its predictive adequacy in accounting for the associated behavior. We show that this new variant, known as the double trace model, has a higher performance in predicting choice data, and shows near optimal reward harvesting efficiency in simulated environments. These results suggests that choice history effects may be adaptive for natural contingencies between consumption and reward availability. This concept lends credence to a normative account of choice history effects that extends beyond its description as a bias.

show abstract

“…by design (Gabay & Apps, 2020;Mormann & Russo, 2021;Yoo et al, 2021). As demonstrated with the learnable version of the foraging paradigm we reviewed earlier (Hall-McMaster et al, 2021), providing people with contextual structure and additional control over their choices provides a new window into how they make them (Constantino & Daw, 2015;Kolling et al, 2012;Shenhav et al, 2016;Yoon, Geary, Ahmed, & Shadmehr, 2018). Taking this route may also shed new light on the computations underlying neural correlates of choice value.…”

Section: Decisions and Control Over Future Research Directionsmentioning

confidence: 98%

“…One example is a recent study that modified the standard foraging paradigm, where people need to decide when to abandon a depleting patch for an unknown new one Shenhav et al, 2016; so that people could revisit previous patches and decide where to forage next. Hall-McMaster, Dayan, and Schuck (2021) showed that people learn and represent reward rates of different patches and use this information to choose patches that maximize reward. Importantly, they also leverage information about the rewards at other patches to adaptively adjust decisions about when to leave the current patch.…”

Section: Monitoring Over Multiple Levels Of a Response Hierarchymentioning

confidence: 99%

Filling the gaps: Cognitive control as a critical lens for understanding mechanisms of value-based decision-making

Frömer

Shenhav

2022

Neuroscience & Biobehavioral Reviews

View full text Add to dashboard Cite

While often seeming to investigate rather different problems, research into value-based decision making and cognitive control have historically offered parallel insights into how people select thoughts and actions. While the former studies how people weigh costs and benefits to make a decision, the latter studies how they adjust information processing to achieve their goals.Recent work has highlighted ways in which decision-making research can inform our understanding of cognitive control. Here, we provide the complementary perspective: how cognitive control research has informed understanding of decision-making. We highlight three particular areas of research where this critical interchange has occurred: (1) how different types of goals shape the evaluation of choice options, (2) how people use control to adjust how they make their decisions, and (3) how people monitor decisions to inform adjustments to control at multiple levels and timescales. We show how adopting this alternate viewpoint offers new insight into the determinants of both decisions and control; provides alternative interpretations for common neuroeconomic findings; and generates fruitful directions for future research.

show abstract

Control over patch encounters changes foraging behaviour

Cited by 4 publications

References 55 publications

Filling the gaps: Cognitive control as a critical lens for understanding mechanisms of value-based decision-making

Filling the gaps: Cognitive control as a critical lens for understanding mechanisms of value-based decision-making

Choice history effects in mice and humans improve reward harvesting efficiency

Filling the gaps: Cognitive control as a critical lens for understanding mechanisms of value-based decision-making

Contact Info

Product

Resources

About