Reward feedback accelerates motor learning

Abstract

Recent findings have demonstrated that reward feedback alone can drive motor learning. However, it is not yet clear whether reward feedback alone can lead to learning when a perturbation is introduced abruptly, or how a reward gradient can modulate learning. In this study, we provide reward feedback that decays continuously with increasing error. We asked whether it is possible to learn an abrupt visuomotor rotation by reward alone, and if the learning process could be modulated by combining reward and sensory feedback and/or by using different reward landscapes. We designed a novel visuomotor learning protocol during which subjects experienced an abruptly introduced rotational perturbation. Subjects received either visual feedback or reward feedback, or a combination of the two. Two different reward landscapes, where the reward decayed either linearly or cubically with distance from the target, were tested. Results demonstrate that it is possible to learn from reward feedback alone and that the combination of reward and sensory feedback accelerates learning. An analysis of the underlying mechanisms reveals that although reward feedback alone does not allow for sensorimotor remapping, it can nonetheless lead to broad generalization, highlighting a dissociation between remapping and generalization. Also, the combination of reward and sensory feedback accelerates learning without compromising sensorimotor remapping. These findings suggest that the use of reward feedback is a promising approach to either supplement or substitute sensory feedback in the development of improved neurorehabilitation techniques. More generally, they point to an important role played by reward in the motor learning process.

Characteristics of Implicit Sensorimotor Adaptation Revealed by Task-irrelevant Clamped Feedback

“…Things are not as clear for the Ignore group, but the contingency between the movement and cursor position may have induced participants, either explicitly or implicitly, to modify their behavior because of perceived performance errors (they were instructed that the feedback would always be rotated by 45°). This is especially problematic as mechanisms associated with reward and punishment have been hypothesized to modulate the rate of learning and retention in visuomotor adaptation tasks (Galea, Mallia, Rothwell, & Diedrichsen, 2015; Nikooyan & Ahmed, 2015). Such a modulation of learning or retention could have affected our measures of implicit adaptation in either group.…”

State-Based Delay Representation and Its Transfer from a Game of Pong to Reaching and Tracking

“…Because participants were instructed to hit the ball as many times as possible within the time duration of each trial, and were provided with a feedback according to this performance measure, we reported their hit rate throughout the experiments. These hits can be considered as reward signals that influence future interception attempts in a reinforcement learning mechanism (Izawa and Shadmehr, 2011; Wolpert et al 2011; Shmuelof et al 2012; Nikooyan and Ahmed, 2015). If the adaptation is error-based (Thoroughman and Shadmehr, 2000; Donchin et al 2003; Smith et al 2006; Herzfeld et al 2014), the candidate error signals need to be identified; for example, the distance between the hand and the paddle at meaningful events during the game such as ball-paddle hits.…”

Robot assistance of motor learning: A neuro-cognitive perspective

“…Later on the distinction between the evaluative and the informational content of feedback was emphasized (e.g., Annett, 1969). The role of these two types of content can be varied by way of manipulating the kind of feedback (e.g., Heuer and Hegele, 2014;Nikooyan and Ahmed, 2015) or by referring the feedback not only to perfect performance, but also to the best performance reached that far (e.g. SansMuntadas et al, 2014).…”