Voluntary motor commands produce two kinds of consequences. Initially, a sensory consequence is observed in terms of activity in our primary sensory organs (e.g., vision, proprioception). Subsequently, the brain evaluates the sensory feedback and produces a subjective measure of utility or usefulness of the motor commands (e.g., reward). As a result, comparisons between predicted and observed consequences of motor commands produce two forms of prediction error. How do these errors contribute to changes in motor commands? Here, we considered a reach adaptation protocol and found that when high quality sensory feedback was available, adaptation of motor commands was driven almost exclusively by sensory prediction errors. This form of learning had a distinct signature: as motor commands adapted, the subjects altered their predictions regarding sensory consequences of motor commands, and generalized this learning broadly to neighboring motor commands. In contrast, as the quality of the sensory feedback degraded, adaptation of motor commands became more dependent on reward prediction errors. Reward prediction errors produced comparable changes in the motor commands, but produced no change in the predicted sensory consequences of motor commands, and generalized only locally. Because we found that there was a within subject correlation between generalization patterns and sensory remapping, it is plausible that during adaptation an individual's relative reliance on sensory vs. reward prediction errors could be inferred. We suggest that while motor commands change because of sensory and reward prediction errors, only sensory prediction errors produce a change in the neural system that predicts sensory consequences of motor commands.

When we use a novel tool, the motor commands may not produce the expected outcome. In healthy individuals, with practice the brain learns to alter the motor commands. This change depends critically on the cerebellum as damage to this structure impairs adaptation. However, it is unclear precisely what the cerebellum contributes to the process of adaptation in human motor learning. Is the cerebellum crucial for learning to associate motor commands with novel sensory consequences, called forward model, or is the cerebellum important for learning to associate sensory goals with novel motor commands, called inverse model? Here, we compared performance of cerebellar patients and healthy controls in a reaching task with a gradual perturbation schedule. This schedule allowed both groups to adapt their motor commands. Following training, we measured two kinds of behavior: in one case people were presented with reach targets near the direction in which they had trained. The resulting generalization patterns of patients and controls were similar, suggesting comparable inverse models. In another case, they reached without a target and reported the location of their hand. In controls the pattern of change in reported hand location was consistent with simulation results of a forward model that had learned to associate motor commands with new sensory consequences. In patients, this change was significantly smaller. Therefore, in our sample of patients we observed that while adaptation of motor commands can take place despite cerebellar damage, cerebellar integrity appears critical for learning to predict visual sensory consequences of motor commands.

Children with autism spectrum disorder (ASD) exhibit deficits in motor control, imitation, and social function. Does a dysfunction in the neural basis of representing internal models of action contribute to these problems? We measured patterns of generalization as children learned to control a novel tool and found that the autistic brain built a stronger than normal association between self generated motor commands and proprioceptive feedback; furthermore, the greater the reliance on proprioception, the greater the child’s impairments in social function and imitation.

Adaptation is sometimes viewed as a process in which the nervous system learns to predict and cancel effects of a novel environment, returning movements to near baseline (unperturbed) conditions. An alternate view is that cancellation is not the goal of adaptation. Rather, the goal is to maximize performance in that environment. If performance criteria are well defined, theory allows one to predict the reoptimized trajectory. For example, if velocity-dependent forces perturb the hand perpendicular to the direction of a reaching movement, the best reach plan is not a straight line but a curved path that appears to overcompensate for the forces. If this environment is stochastic (changing from trial to trial), the reoptimized plan should take into account this uncertainty, removing the overcompensation. If the stochastic environment is zero-mean, peak velocities should increase to allow for more time to approach the target. Finally, if one is reaching through a via-point, the optimum plan in a zero-mean deterministic environment is a smooth movement but in a zero-mean stochastic environment is a segmented movement. We observed all of these tendencies in how people adapt to novel environments. Therefore, motor control in a novel environment is not a process of perturbation cancellation. Rather, the process resembles reoptimization: through practice in the novel environment, we learn internal models that predict sensory consequences of motor commands. Through reward-based optimization, we use the internal model to search for a better movement plan to minimize implicit motor costs and maximize rewards.

Movement variability is often considered an unwanted byproduct of a noisy nervous system. However, variability can signal a form of implicit exploration, indicating that the nervous system is intentionally varying the motor commands in search of actions that yield the greatest success. Here, we investigated the role of the human basal ganglia in controlling reward-dependent motor variability as measured by trial-to-trial changes in performance during a reaching task. We designed an experiment in which the only performance feedback was success or failure and quantified how reach variability was modulated as a function of the probability of reward. In healthy controls, reach variability increased as the probability of reward decreased. Control of variability depended on the history of past rewards, with the largest trial-to-trial changes occurring immediately after an unrewarded trial. In contrast, in participants with Parkinson's disease, a known example of basal ganglia dysfunction, reward was a poor modulator of variability; that is, the patients showed an impaired ability to increase variability in response to decreases in the probability of reward. This was despite the fact that, after rewarded trials, reach variability in the patients was comparable to healthy controls. In summary, we found that movement variability is partially a form of exploration driven by the recent history of rewards. When the function of the human basal ganglia is compromised, the reward-dependent control of movement variability is impaired, particularly affecting the ability to increase variability after unsuccessful outcomes.

Lay Abstract Children with autism spectrum disorder (ASD) show deficits in development of motor skills, in addition to core deficits in social skill development. In a previous study (Haswell et al., 2009) we found that children with autism show a key difference in how they learn motor actions, with a bias for relying on joint position rather than visual feedback; further, this pattern of motor learning predicted impaired motor, imitation and social abilities. We were interested in finding out whether this altered motor learning pattern was specific to autism. To do so, we examined children with Attention Deficit Hyperactivity Disorder (ADHD), who also show deficits in motor control. Children learned a novel movement and we measured rates of motor learning, generalization patterns of motor learning, and variability of motor speed during learning. We found children with ASD show a slower rate of learning and, consistent with previous findings, an altered pattern of generalization that was predictive of impaired motor, imitation, and social impairment. In contrast, children with ADHD showed a normal rate of learning and a normal pattern of generalization; instead, they (and they alone), showed excessive variability in movement speed. The findings suggest that there is a specific pattern of altered motor learning associated with autism. Scientific Abstract The brain builds an association between action and sensory feedback to predict the sensory consequence of self-generated motor commands. This internal model of action is central to our ability to adapt movements, and may also play a role in our ability to learn from observing others. Recently we reported that the spatial generalization patterns that accompany adaptation of reaching movements were distinct in children with Autism Spectrum Disorder (ASD) as compared to typically developing (TD) children. To test whether the generalization patterns are specific to ASD, here we compared the patterns of adaptation to those in children with Attention Deficit Hyperactivity Disorder (ADHD). Consistent with our previous observations, we found that in ASD the motor memory showed greater than normal generalization in proprioceptive coordinates compared with both TD children and children with ADHD; children with ASD also showed slower rates of adaptation compared with both control groups. Children with ADHD did not show this excessive generalization to the proprioceptive target, but did show excessive variability in the speed of movements with an increase in the exponential distribution of responses (τ) as compared with both TD children and children with ASD. The results suggest that slower rate of adaptation and anomalous bias towards proprioceptive feedback during motor learning is characteristic of autism; whereas increased variability in execution is characteristic of ADHD.

Our sensory observations represent a delayed, noisy estimate of the environment. Delay causes instability and noise causes uncertainty. To deal with these problems, theory suggests that the processing of sensory information by the brain should be probabilistic: to start a movement or to alter it midflight, our brain should make predictions about the near future of sensory states and then continuously integrate the delayed sensory measures with predictions to form an estimate of the current state. To test the predictions of this theory, we asked participants to reach to the center of a blurry target. With increased uncertainty about the target, reach reaction times increased. Occasionally, we changed the position of the target or its blurriness during the reach. We found that the motor response to a given second target was influenced by the uncertainty about the first target. The specific trajectories of motor responses were consistent with predictions of a "minimum variance" state estimator. That is, the motor output that the brain programmed to start a reaching movement or correct it midflight was a continuous combination of two streams of information: a stream that predicted the near future of the state of the environment and a stream that provided a delayed measurement of that state.

It is widely accepted that the cerebellum acquires and maintain internal models for motor control. An internal model simulates mapping between a set of causes and effects. There are two candidates of cerebellar internal models, forward models and inverse models. A forward model transforms a motor command into a prediction of the sensory consequences of a movement. In contrast, an inverse model inverts the information flow of the forward model. Despite the clearly different formulations of the two internal models, it is still controversial whether the cerebro-cerebellum, the phylogenetically newer part of the cerebellum, provides inverse models or forward models for voluntary limb movements or other higher brain functions. In this article, we review physiological and morphological evidence that suggests the existence in the cerebro-cerebellum of a forward model for limb movement. We will also discuss how the characteristic input-output organization of the cerebro-cerebellum may contribute to forward models for non-motor higher brain functions.