The role of executive function in shaping reinforcement learning

Rmus, Milena; McDougle, Samuel D.; Collins, Anne

doi:10.1016/j.cobeha.2020.10.003

Cited by 44 publications

(44 citation statements)

References 68 publications

(82 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Furthermore, many aspects of learning from reward do not depend on the brain's RL network, whether they are captured by RL algorithms or not. For example, hippocampal episodic memory [24,25] and prefrontal working memory [26,27,28] contribute to RL behavior, but are often not explicitly modeled in RL, obscuring the contribution of non-RL neural processes to learning.…”

Section: Rl In Machine Learning Psychology and Neurosciencementioning

confidence: 99%

“…These links are often assumed to be specific one-to-one mappings: "Dopamine neurons code an error in the prediction of reward" [20]; "corticostriatal loops enable state-dependent value-based choice" [27]; "striatal areas [...] support reinforcement learning, and frontoparietal attention areas [...] support executive control processes" [42]; "individual differences in DA clearance and frontostriatal coordination may serve as markers for RL" [43]; and "BOLD activity in the VS, dACC, and vmPFC is correlated with learning rate, expected value, and prediction error, respectively" [44]. This shows that computational variables are often interpreted as specific (neuro)cognitive functions, revealing an assumption of interpretability.…”

Section: Interpretability and Generalizabilitymentioning

confidence: 99%

“…Text translation, Robotics [85] Distributional RL [39] Dopamine reward prediction errors [19], Hierarchical RL [12] Working memory [25,26,27],…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

What do reinforcement learning models measure? Interpreting model parameters in cognition and neuroscience

Eckstein

Wilbrecht

Collins

2021

Current Opinion in Behavioral Sciences

Self Cite

View full text Add to dashboard Cite

Reinforcement Learning" (RL) refers to different concepts in machine learning, psychology, and neuroscience.• In psychology and neuroscience, RL models have provided successful methods for describing and predicting complex behavioral processes and brain activity.• However, RL variables often do not generalize from one task to another and need to be interpreted in context.• RL computations also do not always reflect the same underlying cognitive and neural processes.• A more nuanced understanding of RL and its variables (e.g., reward prediction errors, parameters) is necessary to move the field forward.

show abstract

Section: Rl In Machine Learning Psychology and Neurosciencementioning

confidence: 99%

Section: Interpretability and Generalizabilitymentioning

confidence: 99%

See 1 more Smart Citation

What do reinforcement learning models measure? Interpreting model parameters in cognition and neuroscience

Eckstein

Wilbrecht

Collins

2021

Current Opinion in Behavioral Sciences

Self Cite

View full text Add to dashboard Cite

show abstract

“…Our hypothesis is also inspired by recent research demonstrating that top-down inputs directly influence value-based learning computations in RL circuits (Rmus et al, 2021). For instance, attention modulates RL by specifying reward-predicting features of stimuli that RL should operate on (Leong et al, 2017; Radulescu et al, 2019).…”

Section: Introductionmentioning

confidence: 93%

Executive function supports single-shot endowment of value to arbitrary transient goals

McDougle

Ballard

Baribault³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Recent evidence suggests that executive processes shape reinforcement learning (RL) computations. Here, we extend this idea to the processing of choice outcomes, asking if executive function and RL interact during learning from novel goals. We designed a task where people learned from familiar rewards or abstract instructed goals. We hypothesized that learning from these goals would produce reliable responses in canonical reward circuits, and would do so by leveraging executive function. Behavioral results pointed to qualitatively similar learning processes when subjects learned from achieving goals versus familiar rewards. Goal learning was robustly and selectively correlated with performance on an independent executive function task. Neuroimaging revealed comparable appetitive responses and computational signatures in reinforcement learning circuits for both goal-based and familiar learning contexts. During goal learning, we observed enhanced correlations between prefrontal cortex and canonical reward-sensitive regions, including hippocampus, striatum, and the midbrain. These findings demonstrate that attaining novel goals produces reliable reward signals in dopaminergic circuits. We propose that learning from goal-directed behavior is mediated by top-down input that primes the reward system to endow value to cues signaling goal attainment.

show abstract

“…Reinforcement Learning (RL) is inspired by the behavioral psychology and it pays attention to how to take action so as to furthest increase the accumulated reward in terms of the agent 2 . To be specific, RL is composed of agent, environment, state, action and reward 3 .…”

Section: Introductionmentioning

confidence: 99%

ICN‐driven group psychology visualization analysis mechanism using reinforcement learning

Qin

2021

Internet Technology Letters

View full text Add to dashboard Cite

Reinforcement Learning (RL) has been widely considered as a robust method to complete the large‐scale data analysis with high computation efficiency, learning ability, and stability, and it has been applied into many fields, such as transaction detection, trajectory identification, behavior recognition and so on. With the rapid progress of human society, people's psychological pressure gradually becomes heavier and heavier. In order to avoid the social tragedy, it is certainly worth doing the group psychology analysis. Therefore, this paper uses RL to achieve the group psychology visualization analysis instead of the abstract data presentation. In other words, this paper also handles the analyzed data results on the group psychology behaviors by the visualization method. In addition, in order to accelerate the process of data visualization under the large‐scale environment, this paper also introduces the concept of Information‐Centric Networking (ICN) paradigm to support the name‐based community detection by separating IP addresses. The experiments include two parts. On one hand, RL‐based group psychology analysis method is evaluated. On the other hand, ICN‐based visualization method is evaluated. The results prove that the integrated group psychology visualization analysis mechanism is acceptable.

show abstract

The role of executive function in shaping reinforcement learning

Cited by 44 publications

References 68 publications

What do reinforcement learning models measure? Interpreting model parameters in cognition and neuroscience

What do reinforcement learning models measure? Interpreting model parameters in cognition and neuroscience

Executive function supports single-shot endowment of value to arbitrary transient goals

ICN‐driven group psychology visualization analysis mechanism using reinforcement learning

Contact Info

Product

Resources

About