Models and Methods for Reinforcement Learning

Dayan, Peter; Nakahara, Hiroyuki

doi:10.1002/9781119170174.epcn513

Cited by 2 publications

(1 citation statement)

References 152 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The reinforcement learning framework has provided a foundation for understanding the computations and associated neural mechanisms involved in self-regarding valuation (Schultz et al, 1997;Daw and Doya, 2006;Dayan and Nakahara, 2018). It provides a simple, rigorous account of choice behavior that maxi-mizes one's own reward.…”

Section: Introductionmentioning

confidence: 99%

Computing Social Value Conversion in the Human Brain

Fukuda

Suzuki

et al. 2019

J. Neurosci.

Self Cite

View full text Add to dashboard Cite

Social signals play powerful roles in shaping self-oriented reward valuation and decision making. These signals activate social and valuation/decision areas, but the core computation for their integration into the self-oriented decision machinery remains unclear. Here, we study how a fundamental social signal, social value (others' reward value), is converted into self-oriented decision making in the human brain. Using behavioral analysis, modeling, and neuroimaging, we show three-stage processing of social value conversion from the offer to the effective value and then to the final decision value. First, a value of others' bonus on offer, called offered value, was encoded uniquely in the right temporoparietal junction (rTPJ) and also in the left dorsolateral prefrontal cortex (ldlPFC), which is commonly activated by offered self-bonus value. The effective value, an intermediate value representing the effective influence of the offer on the decision, was represented in the right anterior insula (rAI), and the final decision value was encoded in the medial prefrontal cortex (mPFC). Second, using psychophysiological interaction and dynamic causal modeling analyses, we demonstrated three-stage feedforward processing from the rTPJ and ldPFC to the rAI and then from rAI to the mPFC. Further, we showed that these characteristics of social conversion underlie distinct sociobehavioral phenotypes. We demonstrate that the variability in the conversion underlies the difference between prosocial and selfish subjects, as seen from the differential strength of the rAI and ldlPFC coupling to the mPFC responses, respectively. Together, these findings identified fundamental neural computation processes for social value conversion underlying complex social decision making behaviors.

show abstract

Section: Introductionmentioning

confidence: 99%

Computing Social Value Conversion in the Human Brain

Fukuda

Suzuki

et al. 2019

J. Neurosci.

Self Cite

View full text Add to dashboard Cite

show abstract

Reinforcement Learning in Spiking Neural Networks with Stochastic and Deterministic Synapses

Yuan

Yan

et al. 2019

Neural Computation

View full text Add to dashboard Cite

Though succeeding in solving various learning tasks, most existing reinforcement learning (RL) models have failed to take into account the complexity of synaptic plasticity in the neural system. Models implementing reinforcement learning with spiking neurons involve only a single plasticity mechanism. Here, we propose a neural realistic reinforcement learning model that coordinates the plasticities of two types of synapses: stochastic and deterministic. The plasticity of the stochastic synapse is achieved by the hedonistic rule through modulating the release probability of synaptic neurotransmitter, while the plasticity of the deterministic synapse is achieved by a variant of a reward-modulated spike-timing-dependent plasticity rule through modulating the synaptic strengths. We evaluate the proposed learning model on two benchmark tasks: learning a logic gate function and the 19-state random walk problem. Experimental results show that the coordination of diverse synaptic plasticities can make the RL model learn in a rapid and stable form.

show abstract

Models and Methods for Reinforcement Learning

Cited by 2 publications

References 152 publications

Computing Social Value Conversion in the Human Brain

Computing Social Value Conversion in the Human Brain

Reinforcement Learning in Spiking Neural Networks with Stochastic and Deterministic Synapses

Contact Info

Product

Resources

About