Modulation of saccadic eye movements by predicted reward outcome

Abstract

Reward is a primary goal of behavior and is crucial for survival of animals. To explore the mechanisms underlying such reward-oriented behavior, we devised a memory-guided saccade task in which only one fixed direction out of four was rewarded, which was called the one-direction-rewarded task (1DR). As the rewarded direction was changed in four blocks, saccades in a given direction were rewarded in one block (constituting reward-oriented behavior), but non-rewarded in the other blocks (non-reward-oriented behavior). As a control, an all-directions-rewarded task (ADR) was used. Using these tasks, we found that the parameters of saccades changed depending on whether or not the saccade was followed by reward. (1) The mean saccadic peak velocity was higher and the mean saccade latency was shorter in the rewarded condition than in the non-rewarded condition. (2) The mean saccade amplitude showed no difference in two out of three monkeys. (3) The variations of saccadic velocity, latency and amplitude were smaller in the rewarded condition. (4) Within a block of 1DR, the saccade velocity remained high in the rewarded condition, but decreased gradually in the non-rewarded condition; it decreased only slightly in ADR. The saccade latency showed the opposite pattern of change, but less clearly. (5) The saccades in the non-rewarded condition tended to have slower velocities and longer latencies in the trials shortly after a rewarded trial. (6) The ratio of error trials was much higher in the non-rewarded condition than the rewarded condition. (7) The errors, which were due to premature or incorrect saccades, showed unique spatiotemporal patterns that would reflect the competition between the cognitive and motivational processes. These results provide important constraints to the neuronal mechanism underlying reward-oriented behavior because it must satisfy these rules.

Inside the brain of an elite athlete: the neural processes that support high achievement in sports

“…An example of the latter is the observation that monkeys make faster and less variable saccades to those targets associated with the most reward 114 . Recent developments in reinforcement learning suggest that task-specific rewards may operate through increased dopaminedependent weighting of 'teaching signals' (phasic dopaminergic signals thought to represent the reward prediction error: the difference between the expected and actual reward in a given trial or time step).…”

Adaptive Control of Saccades via Internal Feedback

“…One component of the cost may be a measure of endpoint variability Wolpert, 1998, 2006). Another component may be the value that the brain assigns to the visual stimulus, because rewarded stimuli tend to produce faster saccades (Takikawa et al, 2002). A balance between these factors nicely reproduces the velocities and durations that one typically observes in saccades to stationary targets.…”

“…In the optimal control framework, planning of any action takes into account the value associated with the stimulus: the movement is slower toward a stimulus that has a lower value. Indeed, when one varies the amount of reward associated with a visual stimulus, saccade velocities are faster for stimuli with higher expected reward (Takikawa et al, 2002). It is possible that when a visual stimulus repeats, the internal value that the brain associates with it becomes smaller.…”

Evidence integration in model-based tree search

“…Both response (31,32) and movement (33,34) vigor (the speeds at which participants initiate and carry out an action) are thought to be modulated by reward. The next set of models test the possibility that the second-stage reaction time effect can be explained based on these principles alone, without the need for continued deliberation.…”