Reward abundance interferes with error-based learning in a visuomotor adaptation task

Abstract

The brain rapidly adapts reaching movements to changing circumstances by using visual feedback about errors. Providing reward in addition to error feedback facilitates the adaptation but the underlying mechanism is unknown. Here, we investigate whether the proportion of trials rewarded (the ‘reward abundance’) influences how much participants adapt to their errors. We used a 3D multi-target pointing task in which reward alone is insufficient for motor adaptation. Participants (N = 423) performed the pointing task with feedback based on a shifted hand-position. On a proportion of trials we gave them rewarding feedback that their hand hit the target. Half of the participants only received this reward feedback. The other half also received feedback about endpoint errors. In different groups, we varied the proportion of trials that was rewarded. As expected, participants who received feedback about their errors did adapt, but participants who only received reward-feedback did not. Critically, participants who received abundant rewards adapted less to their errors than participants who received less reward. Thus, reward abundance negatively influences how much participants learn from their errors. Probably participants used a mechanism that relied more on the reward feedback when the reward was abundant. Because participants could not adapt to the reward, this interfered with adaptation to errors.

Quantifying exploration in reward-based motor learning

“…Using a weight-shifting task, we replicated a finding from studies using arm movements: increased variability following non-rewarded trials as compared to rewarded trials [4,5,[8][9][10][11][12][13]. In addition, we found a ratio between variability due to exploration and sensorimotor noise of 0.78.…”

“…Therefore, this finding implies that studying sensorimotor noise and exploration requires a high number of trials. In the literature, human motor learning experiments usually seem to contain up to 200 feedback trials [8,10,12,[31][32][33][34][35][36], although some studies report higher numbers, up to 900 trials [4,5,11,16,19,37,38]. An example of a study incorporating many trials is [14], in which rats performed 300k trials on average.…”

“…If you want to improve your performance but receive no feedback on error size and direction, you have to explore which actions lead to reward such that rewarded actions can be exploited [1,7]. Indeed, studies into reward-based motor learning have shown a higher variability in motor output following non-rewarded movements than following rewarded movements [4,5,[8][9][10][11][12][13][14]. Moreover, there are indications that initial variability in motor output, regarded as exploration, is positively related with learning, both in songbirds [15] and in humans [16].…”

“…To eliminate the influence of learning on variability, feedback was stochastic. To verify whether this task results in exploration, we first aimed to replicate the finding that variability following non-rewarded trials is higher than variability following rewarded trials [4,5,[8][9][10][11][12][13][14]. Next, we aimed to compare the estimation of sensorimotor noise in three feedback contexts: (1) trials in a baseline phase without feedback, (2) trials in short blocks without feedback that were flanked by short blocks of feedback trials and (3) rewarded trials in blocks of feedback trials.…”

Intention to learn modulates the impact of reward and punishment on sequence learning

“…This pattern of behavior is consistent with a win-stay, lose shift strategy 34,35 . Notably, it has been shown that reward is not effective in shaping motor behaviors unless participants are aware of the manipulation being rewarded or if reward is too abundant 36,37 , probably due to participants not engaging in exploratory behaviors. Future work may also consider the psychological implications of reward and punishment given during learning, and how this interacts with behavioral variability and cognitive strategies.…”