Reward-based motor adaptation can generalize across actions.

Abstract

Recently it has been shown that rewarded variability can be used to adapt visuomotor behavior. However, its relevance seems limited because adaptation to binary rewards has been demonstrated only when the same movement is repeated throughout the experiment. We therefore investigated whether the adaptation is action-specific and whether the amount of exploration depends on spatial complexity. Participants pointed to 3-D visual targets without seeing their hand and could use only binary reward feedback to adapt their movements. We varied the number of target positions and the number of dimensions the feedback was based on. Because the feedback was based on a 5-cm rightward shifted hand position, adaptation was needed for good performance. The participants started naïve to the perturbation. If actions were made toward a single target position and the feedback was based on the lateral component of their response only, participants adapted completely within 200 trials. Having more than 1 target position or more than 1 dimension of performance resulted in considerably less adaptation but did not affect the exploration. Thus, reward-based adaptation can generalize across actions but is reduced by spatial complexity, whereas exploration is not affected by spatial complexity. (PsycINFO Database Record

Quantifying exploration in reward-based motor learning

“…Using a weight-shifting task, we replicated a finding from studies using arm movements: increased variability following non-rewarded trials as compared to rewarded trials [4,5,[8][9][10][11][12][13]. In addition, we found a ratio between variability due to exploration and sensorimotor noise of 0.78.…”

“…We estimated sensorimotor noise by assuming that following non-feedback and rewarded trials, participants have no reason to explore [9]. In this way, we could obtain sensorimotor noise estimates in three feedback contexts: following non-feedback trials in the baseline phase, following non-feedback trials in the intermittent phase, and following rewarded trials in the feedback blocks of the intermittent phase.…”

“…If you want to improve your performance but receive no feedback on error size and direction, you have to explore which actions lead to reward such that rewarded actions can be exploited [1,7]. Indeed, studies into reward-based motor learning have shown a higher variability in motor output following non-rewarded movements than following rewarded movements [4,5,[8][9][10][11][12][13][14]. Moreover, there are indications that initial variability in motor output, regarded as exploration, is positively related with learning, both in songbirds [15] and in humans [16].…”

“…To eliminate the influence of learning on variability, feedback was stochastic. To verify whether this task results in exploration, we first aimed to replicate the finding that variability following non-rewarded trials is higher than variability following rewarded trials [4,5,[8][9][10][11][12][13][14]. Next, we aimed to compare the estimation of sensorimotor noise in three feedback contexts: (1) trials in a baseline phase without feedback, (2) trials in short blocks without feedback that were flanked by short blocks of feedback trials and (3) rewarded trials in blocks of feedback trials.…”

Practicing one thing at a time: the secret to reward-based learning?

“…For instance, Katy's dance move will only be successful if she gets both the position and timing right. We previously showed that learning of a visuomotor perturbation in a three-dimensional pointing task was not possible when participants received feedback based on a three-dimensional position error 10-12 . However, learning did occur when the feedback was based on the perturbed dimension only 10 .…”