Behavioural and neural characterization of optimistic reinforcement learning

Lefebvre, Germain; Lebreton, Maël; Meyniel, Florent; Bourgeois‐Gironde, Sacha; Palminteri, Stefano

doi:10.1038/s41562-017-0067

Cited by 220 publications

(309 citation statements)

References 49 publications

Supporting

Mentioning

267

Contrasting

Order By: Relevance

“…Furthermore, participants showed no sign of leaky inference within each sequence, in either condition. The slower reversal learning in the outcome-based condition is also unlikely to arise from a biased, choice-supportive filtering of evidence described and reported across cognitive domains (Sharot et al, 2011;Bronfman et al, 2015;Lefebvre et al, 2017;Talluri et al, 2018;Yon et al, 2018). Indeed, the strength of conflicting evidence (inconsistent with beliefs) could be decoded with equal precision across conditions.…”

Section: Neural Dissociation Between Absolute and Relational Coding Omentioning

confidence: 82%

Interacting with volatile environments stabilizes hidden-state inference and its brain signatures

Weiss

Chambon

Drugowitsch

et al. 2019

Preprint

View full text Add to dashboard Cite

words)Making accurate decisions in uncertain environments requires identifying the generative cause of sensory cues, but also the expected outcomes of possible actions. Although both cognitive processes can be formalized as Bayesian inference, they are commonly studied using different experimental frameworks, making their formal comparison difficult. Here, by framing a reversal learning task either as cue-based or outcome-based inference, we found that humans perceive the same volatile environment as more stable when inferring its hidden state by interaction with uncertain outcomes than by observation of equally uncertain cues. Time-resolved analyses of magnetoencephalographic (MEG) signals explained this behavioral difference by the neural interaction between inferred beliefs and incoming evidence, an effect originating from the medial temporal lobe (MTL).Together, these findings indicate that the degree of control over the sampling of volatile environments shapes human learning and decision-making under uncertainty.

show abstract

Section: Neural Dissociation Between Absolute and Relational Coding Omentioning

confidence: 82%

Interacting with volatile environments stabilizes hidden-state inference and its brain signatures

Weiss

Chambon

Drugowitsch

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…While standard accounts of belief (or value) prescribe that an agent should learn equally well from positive and negative information (e.g., Barto and Sutton, 1998;Benjamin, 2018 for a review), previous studies have consistently shown that people exhibit valence-induced biases (e.g., Lefebvre et al, 2017;Kuzmanovic et al, 2018). These biases are traditionally exemplified in the so-called "good news/bad news" effect whereby people tend to overweight good news relative to bad news when updating self-relevant beliefs (Sharot and Garrett, 2016).…”

Section: Discussionmentioning

confidence: 99%

Choosing and learning: outcome valence differentially affects learning from free versus forced choices

Chambon

Théro

Vidal

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

Positivity bias refers to learning more from positive than negative events. This learning asymmetry could either reflect a preference for positive events in general, or be the upshot of a more general, and perhaps, ubiquitous, "choice-confirmation" bias, whereby agents preferentially integrate information that confirms their previous decision. We systematically compared these two theories with 3 experiments mixing free-and forced-choice conditions, featuring factual and counterfactual learning and varying action requirements across "go" and "no-go" trials. Computational analyses of learning rates showed clear and robust evidence in favour of the "choice-confirmation" theory: participants amplified positive prediction errors in free-choice conditions while being valence-neutral on forced-choice conditions. We suggest that a choice-confirmation bias is adaptive to the extent that it reinforces actions that are most likely to meet an individual's needs, i.e. freely chosen actions. In contrast, outcomes from unchosen actions are more likely to be treated impartially, i.e. to be assigned no special value in self-determined decisions.

show abstract

“…The development, adaptation and evolution of human social learning and cognitive skills in the economic environment is based on experiences acquired from environmental stimulus through time. These skills are crucial for setting perception, preferences and the decisionmaking process of humans interacting into the economic environment (Baker et al, 2017;Bechara and Damasio, 2005;de Bot et al, 2007;Frederick, 2005;Fryer and Jackson, 2003;Glowacki and Molleman, 2017;Kenrick et al, 2009;Lefebvre et al, 2017;Smith, 1991;Vriend, 1996).…”

Section: Resultsmentioning

confidence: 99%

Determining the chaotic behaviour of copper prices in the long-term using annual price data

et al. 2018

View full text Add to dashboard Cite

Mineral commodity prices are influenced by economic, technological, psychological, and geopolitical factors. Stochastic approaches, and time series and econometric techniques have been used to represent the dynamics of mineral commodity markets and predict prices. However, these techniques cannot provide a comprehensive representation of market dynamics because they do not recognise the relationship between these factors over time, and they are unable to capture both the evolution and the cumulative effects of these factors on prices. Stability of motion and chaos theories can detect sensitivity to initial conditions, and therefore the evolutionary patterns allowing a proper understanding and representation of mineral commodity market dynamics. Most of the techniques used to assess chaos require a colossal amount of data, so the use of small data sets to assess chaos has been largely criticised. Nevertheless, by definition, the dynamics of a chaotic system remain at different scales owing to its self-organisation features that exhibit ordered patterns in the absence of codes or rules. Therefore, any deterministic chaotic behaviour of mineral commodity prices can be captured by using small data sets if a detailed qualitative and quantitative analysis are carried out. This paper examines the chaotic behaviour of annual copper prices between 1900 and 2015. To do so, we combine chaos theory, stability of motion and statistical techniques to reconstruct the long-term dynamics of copper prices. First, we examine the time dependency and the presence of a strange attractor by a visual analysis of the time series and phase space reconstruction based on Takens' theorem and determine embedding parameters. Then we examine the dynamic characteristics of the system which assesses its complexity and regularity patterns to measure the system's entropy. Finally, we calculate the largest Lyapunov exponent λ to assess the sensitivity to initial conditions and determine chaotic behaviour supported by a surrogate test. We find that annual copper prices have a chaotic behaviour embedded in a high-dimensional space and short time delay. The study suggests that copper prices exhibit only a single state of low prices, which fluctuate through transitional periods of high prices. It challenges the assertion that metal markets have fluctuated over four major super cycles and debate the adequacy of stochastic and econometric models for representing mineral commodity market behaviour. This study recommends that the use of chaotic behaviour improves our understanding of mineral commodity markets and narrows the data searching, processing and monitoring requirements for forecasting. Therefore, it improves the performance of traditional techniques for selecting key factors that influence the market dynamics, and may also be used to select the most suitable algorithm for forecasting prices.

show abstract

Behavioural and neural characterization of optimistic reinforcement learning

Cited by 220 publications

References 49 publications

Interacting with volatile environments stabilizes hidden-state inference and its brain signatures

Interacting with volatile environments stabilizes hidden-state inference and its brain signatures

Choosing and learning: outcome valence differentially affects learning from free versus forced choices

Determining the chaotic behaviour of copper prices in the long-term using annual price data

Contact Info

Product

Resources

About