Goals, usefulness and abstraction in value-based choice

Martino, Benedetto De; Cortese, Aurelio

doi:10.1016/j.tics.2022.11.001

Cited by 25 publications

(32 citation statements)

References 133 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the one hand, this formalizes the regulating role of confidence in decision making, which has recently been empirically demonstrated in the context of perceptual evidence accumulation 10,11 . On the other hand, this apparently contrasts with standard treatments of value-based decision making, which insists on equating the benefit of value-based decisions with the value of the chosen option [12][13][14] . This notion is a priori appealing, because the purpose of investing resources into decisions is reducible to approaching reward and/or avoiding losses/punishments.…”

Section: Introductionmentioning

confidence: 94%

“…Importantly, although R is analogous to a reward, it is distinct from the values that are attached to the choice options. This does not mean that the values that decision systems attach to choice options are independent from the goal: recent research has demonstrated that option values are strongly influenced by how useful choice options are for achieving one's goal 12,61 . However, at least in principle, alternative choice options that would be instrumental for attaining an important goal may still have low value.…”

Section: On Extending MCD With Goal Hierarchiesmentioning

confidence: 99%

See 1 more Smart Citation

The online metacognitive control of decisions

Lee

Bénon²,

Hopper³

et al. 2023

Preprint

View full text Add to dashboard Cite

Difficult decisions typically involve mental effort, which signals the subjective cost of processing decision-relevant information. But how does the brain regulate mental effort? A possibility is that the brain optimizes a resource allocation problem, whereby the amount of invested resources optimally balances its expected cost (i.e. effort) and benefit. Our working assumption is that subjective decision confidence serves as the benefit term of the resource allocation problem, hence the "metacognitive" nature of decision control (Lee & Daunizeau, 2021). In this work, we present a computational model for the online metacognitive control of decisions or oMCD. Formally, oMCD is a Markov Decision Process that optimally solves the ensuing resource allocation problem under agnostic assumptions about the inner workings of the underlying decision system. We disclose its main properties, when coupled with two standard decision processes (namely: the ideal observer and the attribute integration cases). Importantly, we show that oMCD reproduces most established empirical results in the field of value-based decision making. Finally, we discuss the possible connexions of the model with most prominent neurocognitive theories about mental effort, and highlight potential extensions.

show abstract

Section: Introductionmentioning

confidence: 94%

Section: On Extending MCD With Goal Hierarchiesmentioning

confidence: 99%

The online metacognitive control of decisions

Lee

Bénon²,

Hopper³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…A promising future direction would be to test to what extent these control strategies apply to diverse types of aversive outcomes (e.g., shocks), which would provide a richer understanding of the generalizability of our model's normative predictions of mental effort allocation across diverse incentives (Crawford et al, 2020;Kray et al, 2018). These findings further provide novel insight into how the congruency between expected outcomes and behavioral goals determine our actions (and computational strategies) in value-based decision-making tasks (De Martino & Cortese, 2023;Frömer et al, 2019;Guitart-Masip et al, 2014;Molinaro & Collins, 2023). Consistent with recent work on this topic, our data show that whether our actions are aligned with our goals matters as much (if not more) than the valence of the outcomes.…”

Section: Discussionmentioning

confidence: 89%

Motivational context determines the impact of aversive outcomes on mental effort allocation

Fahey,

Yee,

Leng

et al. 2023

Preprint

View full text Add to dashboard Cite

It is well known that people will exert effort on a task if sufficiently motivated, but how they distribute these efforts across different strategies (e.g., efficiency vs. caution) remains uncertain. Past work has shown that people invest effort differently for potential positive outcomes (rewards) versus potential negative outcomes (penalties). However, this research failed to account for differences in the context in which negative outcomes motivate someone - either as punishment or reinforcement. It is therefore unclear whether effort profiles differ as a function of outcome valence, motivational context, or both. Using computational modeling and our novel Multi-Incentive Control Task, we show that the influence of aversive outcomes on one’s effort profile is entirely determined by their motivational context. Participants (N:91) favored increased caution in response to larger penalties for incorrect responses, and favored increased efficiency in response to larger reinforcement for correct responses, whether positively or negatively incentivized.Statement of RelevancePeople have to constantly decide how to allocate their mental effort, and in doing so can be motivated by both the positive outcomes that effort accrues and the negative outcomes that effort avoids. For example, someone might persist on a project for work in the hopes of being promoted or to avoid being reprimanded or even fired. Understanding how people weigh these different types of incentives is critical for understanding variability in human achievement as well as sources of motivational impairments (e.g., in major depression). We show that people not only consider both potential positive and negative outcomes when allocating mental effort, but that the profile of effort they engage under negative incentives differs depending on whether that outcome is contingent on sustaining good performance (negative reinforcement) or avoiding bad performance (punishment). Clarifying the motivational factors that determine effort exertion is an important step for understanding motivational impairments in psychopathology.

show abstract

“…Additionally, what counts as a change in the environment might differ between modules. We suggest that equipping modules with learnable attention masks (see Methods) allowed them to ignore changes in the environment that were irrelevant to their particular goal; this raises the interesting possibility that empirical learning phenomena, such as latent inhibition (54) and/or goal-conditioned attention (55), may reflect a similar form of learned inattention in the service coping with non-stationarity.…”

Section: Discussionmentioning

confidence: 99%

“…environment that were irrelevant to their particular goal; this raises the interesting possibility that empirical learning phenomena, such as latent inhibition (54) and/or goal-conditioned attention (55), may reflect a similar form of learned inattention in the service coping with non-stationarity.…”

Section: Connections To Reinforcement Learningmentioning

confidence: 99%

Having “multiple selves” helps learning agents explore and adapt in complex changing worlds

Dulberg

Dubey

Berwian

et al. 2022

Preprint

View full text Add to dashboard Cite

Satisfying a variety of conflicting needs in a changing environment is a fundamental challenge for any adaptive agent. Here, we show that designing an agent in a modular fashion as a collection of sub-agents, each dedicated to a separate need, powerfully enhanced the agent's capacity to satisfy its overall needs. We used the formalism of deep reinforcement learning to investigate a biologically relevant multi-objective task: continually maintaining homeostasis of a set of physiologic variables. We then conducted simulations in a variety of environments and compared how modular agents performed relative to standard monolithic agents (i.e., agents that aimed to satisfy all needs in an integrated manner using a single aggregate measure of success). Simulations revealed that modular agents: a) exhibited a form of exploration that was intrinsic and emergent rather than extrinsically imposed; b) were robust to changes in non-stationary environments, and c) scaled gracefully in their ability to maintain homeostasis as the number of conflicting objectives increased. Supporting analysis suggested that the robustness to changing environments and increasing numbers of needs were due to intrinsic exploration and efficiency of representation afforded by the modular architecture. These results suggest that the normative principles by which agents have adapted to complex changing environments may also explain why humans have long been described as consisting of 'multiple selves'.

show abstract

Goals, usefulness and abstraction in value-based choice

Cited by 25 publications

References 133 publications

The online metacognitive control of decisions

The online metacognitive control of decisions

Motivational context determines the impact of aversive outcomes on mental effort allocation

Having “multiple selves” helps learning agents explore and adapt in complex changing worlds

Contact Info

Product

Resources

About