An Explicit Strategy Prevails When the Cerebellum Fails to Compute Movement Errors

Taylor, Jordan A.; Klemfuss, Nola; Ivry, Richard B.

doi:10.1007/s12311-010-0201-x

Cited by 177 publications

(184 citation statements)

References 27 publications

Supporting

Mentioning

158

Contrasting

Order By: Relevance

“…In line with this idea, our regression analysis revealed significant reach adaptation in both the No-Control and In-Control Spatial condition (Fig. S4), consistent with previous findings that adaptation is not dependent on task performance but rather is driven solely by sensory prediction error (12,13). The trial-by-trial adaptation in the No-Control condition is especially striking given the instructions emphasized that reward outcomes and movement error feedback were not linked to movement accuracy.…”

Section: Resultssupporting

confidence: 90%

“…Individuals with cerebellar degeneration often exhibit impairments in sensorimotor adaptation, a deficit that has been attributed to an inability to encode and/or use sensory prediction errors (7,8,13). We hypothesized that this impairment might also impact the participants' ability to resolve, perhaps implicitly, the credit assignment problem, at least when feedback suggests that reward requires coordinated movements.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Credit assignment in movement-dependent reinforcement learning

McDougle

Boggess

Crossley

et al. 2016

Proc. Natl. Acad. Sci. U.S.A.

Self Cite

View full text Add to dashboard Cite

When a person fails to obtain an expected reward from an object in the environment, they face a credit assignment problem: Did the absence of reward reflect an extrinsic property of the environment or an intrinsic error in motor execution? To explore this problem, we modified a popular decision-making task used in studies of reinforcement learning, the two-armed bandit task. We compared a version in which choices were indicated by key presses, the standard response in such tasks, to a version in which the choices were indicated by reaching movements, which affords execution failures. In the key press condition, participants exhibited a strong risk aversion bias; strikingly, this bias reversed in the reaching condition. This result can be explained by a reinforcement model wherein movement errors influence decision-making, either by gating reward prediction errors or by modifying an implicit representation of motor competence. Two further experiments support the gating hypothesis. First, we used a condition in which we provided visual cues indicative of movement errors but informed the participants that trial outcomes were independent of their actual movements. The main result was replicated, indicating that the gating process is independent of participants' explicit sense of control. Second, individuals with cerebellar degeneration failed to modulate their behavior between the key press and reach conditions, providing converging evidence of an implicit influence of movement error signals on reinforcement learning. These results provide a mechanistically tractable solution to the credit assignment problem.decision-making | reinforcement learning | sensory prediction error | reward prediction error | cerebellum W hen a diner reaches across the table and knocks over her coffee, the absence of anticipated reward should be attributed to a failure of coordination rather than diminish her love of coffee. Although this attribution is intuitive, current models of decision-making lack a mechanistic explanation for this seemingly simple computation. We set out to ask if, and how, selection processes in decision-making incorporate information specific to action execution and thus solve the credit assignment problem that arises when an expected reward is not obtained because of a failure in motor execution.Humans are highly capable of tracking the value of stimuli, varying their behavior on the basis of reinforcement history (1, 2), and exhibiting sensitivity to intrinsic motor noise when reward outcomes depend on movement accuracy (3-5). In real-world behavior, the underlying cause of unrewarded events is often ambiguous: A lost point in tennis could occur because the player made a poor choice about where to hit the ball or failed to properly execute the stroke. However, in laboratory studies of reinforcement learning, the underlying cause of unrewarded events is typically unambiguous, either solely dependent on properties of the stimulus or on motor noise. Thus, it remains unclear how people assign credit to either extrins...

show abstract

Section: Resultssupporting

confidence: 90%

Section: Resultsmentioning

confidence: 99%

Credit assignment in movement-dependent reinforcement learning

McDougle

Boggess

Crossley

et al. 2016

Proc. Natl. Acad. Sci. U.S.A.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The cerebellum has long been considered the site of implicit error-based learning as patients with cerebellar damage show impairments in adaptation to split-belt walking (Morton and Bastian 2006), throwing with prisms (Martin et al 1996;Weiner et al 1983), force-field reaching (Rabe et al 2009;Smith and Shadmehr 2005), and visuomotor rotation reaching (Gibo et al 2013;Izawa et al 2012;Rabe et al 2009;Taylor et al 2010). However, neural circuits involved in explicit motor learning are still unknown.…”

Section: Discussionmentioning

confidence: 99%

Blocking trial-by-trial error correction does not interfere with motor learning in human walking

Long

Roemmich

Bastian

2016

Journal of Neurophysiology

View full text Add to dashboard Cite

Long AW, Roemmich RT, Bastian AJ. Blocking trial-by-trial error correction does not interfere with motor learning in human walking. J Neurophysiol 115: 2341-2348, 2016. First published February 24, 2016 doi:10.1152/jn.00941.2015.-Movements can be learned implicitly in response to new environmental demands or explicitly through instruction and strategy. The former is often studied in an environment that perturbs movement so that people learn to correct the errors and store a new motor pattern. Here, we demonstrate in human walking that implicit learning of foot placement occurs even when an explicit strategy is used to block changes in foot placement during the learning process. We studied people learning a new walking pattern on a split-belt treadmill with and without an explicit strategy through instruction on where to step. When there is no instruction, subjects implicitly learn to place one foot in front of the other to minimize step-length asymmetry during split-belt walking, and the learned pattern is maintained when the belts are returned to the same speed, i.e., postlearning. When instruction is provided, we block expression of the new foot-placement pattern that would otherwise naturally develop from adaptation. Despite this appearance of no learning in foot placement, subjects show similar postlearning effects as those who were not given any instruction. Thus locomotor adaptation is not dependent on a change in action during learning but instead can be driven entirely by an unexpressed internal recalibration of the desired movement. adaptation; feedback; gait; motor learning; walking THE HUMAN NERVOUS SYSTEM USES a repertoire of learning mechanisms to change walking patterns to account for a variety of environments and situations. For example, we explicitly think about where to place our feet when stepping on stones in a river but adapt more implicitly to walking in snow. We currently do not understand how these explicit and implicit processes interact when they are put in conflict or engaged toward the same learning goal. For example, if an explicit strategy is used to block the expression of implicit learning, does the latter still occur? Here, we studied the interactions between these explicit and implicit processes using a strategy via visual feedback and a split-belt treadmill, respectively.When subjects walk on a split-belt treadmill with one belt moving faster than the other, they learn a new gait pattern over hundreds of steps by changing where and when they place their feet on the treadmill Long et al. 2015;Reisman et al. 2005). When the treadmill belts are then returned to the same speed, postlearning effects are observed such that subjects retain much of the newly learned spatial pattern that then decays over a couple hundred steps (Reisman et al. 2005). This learning is largely implicit, particularly when the perturbation is gradually introduced (Sawers et al. 2013; TorresOviedo and Bastian 2012): subjects are often unaware that there is a perturbation for much of the learning period. Additi...

show abstract

“…This problem has motivated the idea that the motor system employs forward models to anticipate the sensory consequences of actions (Wolpert and Miall 1996). The cerebellum has been hypothesized to play a critical role in this process, both for the control of movement and for sensorimotor learning (Schlerf et al 2013;Taylor et al 2010;Tseng et al 2007).…”

Section: Temporal Constraints On Different Processes For Motor Learningmentioning

confidence: 99%

Delayed feedback during sensorimotor learning selectively disrupts adaptation but not strategy use

Brudner

Kethidi

Graeupner

et al. 2016

Journal of Neurophysiology

Self Cite

103

132

View full text Add to dashboard Cite

Brudner SN, Kethidi N, Graeupner D, Ivry RB, Taylor JA. Delayed feedback during sensorimotor learning selectively disrupts adaptation but not strategy use. J Neurophysiol 115: 1499 -1511, 2016. First published January 20, 2016 doi:10.1152/jn.00066.2015.-In sensorimotor adaptation tasks, feedback delays can cause significant reductions in the rate of learning. This constraint is puzzling given that many skilled behaviors have inherently long delays (e.g., hitting a golf ball). One difference in these task domains is that adaptation is primarily driven by error-based feedback, whereas skilled performance may also rely to a large extent on outcome-based feedback. This difference suggests that error-and outcome-based feedback may engage different learning processes, and these processes may be associated with different temporal constraints. We tested this hypothesis in a visuomotor adaptation task. Error feedback was indicated by the terminal position of a cursor, while outcome feedback was indicated by points. In separate groups of participants, the two feedback signals were presented immediately at the end of the movement, after a delay, or with just the error feedback delayed. Participants learned to counter the rotation in a similar manner regardless of feedback delay. However, the aftereffect, an indicator of implicit motor adaptation, was attenuated with delayed error feedback, consistent with the hypothesis that a different learning process supports performance under delay. We tested this by employing a task that dissociates the contribution of explicit strategies and implicit adaptation. We find that explicit aiming strategies contribute to the majority of the learning curve, regardless of delay; however, implicit learning, measured over the course of learning and by aftereffects, was significantly attenuated with delayed error-based feedback. These experiments offer new insight into the temporal constraints associated with different motor learning processes.forward model; internal model; motor adaptation; motor learning; reinforcement learning LEARNING TO LINK ACTIONS with their respective outcomes is necessary for successfully interacting with the world. Feedback delays can present a significant problem in learning actionoutcome associations, yet such delays are ubiquitous in motor control. Feedback from proprioception and vision are inherently delayed because of neural transmission and integration time. Even if these delays are relatively short, the control system faces a significant problem due to potentially destabilizing effects arising from body and environmental dynamics.One prominent solution for handling delayed feedback in the sensorimotor learning literature centers on the idea of an adaptive forward model, a representation that allows the system to anticipate or predict the sensory consequences of an action through sensory-prediction errors (Wolpert and Miall 1996). However, various lines of evidence indicate that this solution is subject to severe temporal constraints. Modest additional delays ...

show abstract

An Explicit Strategy Prevails When the Cerebellum Fails to Compute Movement Errors

Cited by 177 publications

References 27 publications

Credit assignment in movement-dependent reinforcement learning

Credit assignment in movement-dependent reinforcement learning

Blocking trial-by-trial error correction does not interfere with motor learning in human walking

Delayed feedback during sensorimotor learning selectively disrupts adaptation but not strategy use

Contact Info

Product

Resources

About